BanditFuzz: Fuzzing SMT Solvers with Multi-agent Reinforcement Learning

Scott, Joseph; Sudula, Trishal; Rehman, Hammad; Mora, Federico

doi:10.1007/978-3-030-90870-6_6

Cited by 10 publications

(5 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The closest related work for HornFuzz is research on fuzzing SMT solvers. There are many efficient SMT solver fuzzers: STORM [17], BanditFuzz [25], FuzzSMT [5], Falcon [29], OpFuzz [28], YinYang [27], etc.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

HornFuzz: Fuzzing CHC solvers

Суханова

Sobol

2023

Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering

View full text Add to dashboard Cite

Many advanced program analysis and verification methods are based on solving systems of Constrained Horn Clauses (CHC). Testing CHC solvers is very important, as correctness of their work determines whether bugs in the analyzed programs are detected or missed. One of the well-established and efficient methods of automated software testing is fuzzing: analyzing the reactions of programs to random input data. Currently, there are no fuzzers for CHC solvers, and fuzzers for SMT solvers are not efficient in CHC solver testing, since they do not consider CHC specifics. In this paper, we present HornFuzz, a mutation-based gray-box fuzzing technique for detecting bugs in CHC solvers based on the idea of metamorphic testing. We evaluated our fuzzer on one of the highest performing CHC solvers, Spacer, and found a handful of bugs in Spacer. In particular, some discovered problems are so serious that they require fixes with significant changes to the solver. CCS CONCEPTS• Software and its engineering → Software verification and validation.

show abstract

Section: Related Workmentioning

confidence: 99%

“…One of the fast and efficient approaches to finding bugs is fuzzing: an automated software testing technique that involves providing unexpected or random data as input to a computer program and analyzing the reaction of the program. It is commonly used in the domains of software security and quality assurance [5,17,25,[27][28][29].…”

mentioning

confidence: 99%

HornFuzz: Fuzzing CHC solvers

Суханова

Sobol

2023

Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering

View full text Add to dashboard Cite

show abstract

“…Most recently, Park et al [47] presented TypeFuzz, a hybrid approach for integers, reals, and strings which mutates SMT-LIBv2 by replacing expressions with newly generated expressions. Finally, Scott et al [49] recently proposed a mutational fuzzer for all of SMT-LIB which leverages reinforcement learning and targets performance issues.…”

Section: Contributionsmentioning

confidence: 99%

“…In contrast to some recent mutation-based SMT-LIBv2 input fuzzing approaches [39,49,51,52], the API Fuzzer is generation-based : it generates expressions that, importantly, respect the semantic and API models of the solver under test. Non-leaf terms are generated by combining leaf terms (variables or theory-specific constants) and previously generated terms via any of the enabled operators.…”

Section: Api Fuzzermentioning

confidence: 99%

“…In 2009, Brummayer et al [21] presented a grammar-based generative black-box input fuzzer for the SMT-LIBv1 language [48] called FuzzSMT, and in 2017, Niemetz et al [45] presented a model-based API fuzz testing framework called BtorMBT for the SMT solver Boolector. More recently, fuzz testing of SMT solvers via their textual interface has gained even more traction with a series of papers on the subject in top venues [19,39,47,49,51,52]. Note that these approaches (and this paper) assume full knowledge of the input structure, i.e., they only generate valid textual input or sequences of API calls.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Murxla: A Modular and Highly Extensible API Fuzzer for SMT Solvers

Niemetz

Preiner

Barrett

2022

Computer Aided Verification

View full text Add to dashboard Cite

SMT solvers are highly complex pieces of software with performance, robustness, and correctness as key requirements. Complementing traditional testing techniques for these solvers with randomized stress testing has been shown to be quite effective. Recent work has showcased the value of input fuzzing for finding issues, but this approach typically does not comprehensively test a solver’s API. Previous work on model-based API fuzzing was tailored to a single solver and a small subset of SMT-LIB. We present Murxla, a comprehensive, modular, and highly extensible model-based API fuzzer for SMT solvers. Murxla randomly generates valid sequences of solver API calls based on a customizable API model, with full support for the semantics and features of SMT-LIB. It is solver-agnostic but extensible to allow for solver-specific testing and supports option fuzzing, cross-checking with other solvers, translation to SMT-LIBv2, and SMT-LIBv2 input fuzzing. Our evaluation confirms its efficacy in finding issues in multiple state-of-the-art SMT solvers.

show abstract