Turning programs against each other: high coverage fuzz-testing using binary-code mutation and dynamic slicing

Kargén, Ulf; Shahmehri, Nahid

doi:10.1145/2786805.2786844

Cited by 48 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several other works change mutate to be aware of taint-level observations about the program behavior, specifically mutating inputs that are used by the program [8,10,33,44]. Where other fuzzers use pre-defined data mutation strategies like bit flipping or rand replacement, MutaGen uses fragments of the program under test that parse or manipulate the input as mutators through dynamic slicing [29]. SDF uses properties of the seeds themselves to guide mutation [35].…”

Section: Recent Advances In Fuzzingmentioning

confidence: 99%

“…Every trial was allowed to run for 24 hours, and we generally measured at least 30 trials per configuration. We also considered a variety of seed files, including the empty file, paper benchmarks baseline trials variance crash coverage seed timeout MAYHEM [8] R (29) [44] O means other baseline used by no more than 1 paper. Trials: number of trials.…”

Section: Platform and Configurationmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Fuzz Testing

Klees

Ruef

Cooper

et al. 2018

Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

423

278

View full text Add to dashboard Cite

Fuzz testing has enjoyed great success at discovering security critical bugs in real software. Recently, researchers have devoted significant effort to devising new fuzzing techniques, strategies, and algorithms. Such new ideas are primarily evaluated experimentally so an important question is: What experimental setup is needed to produce trustworthy results? We surveyed the recent research literature and assessed the experimental evaluations carried out by 32 fuzzing papers. We found problems in every evaluation we considered. We then performed our own extensive experimental evaluation using an existing fuzzer. Our results showed that the general problems we found in existing experimental evaluations can indeed translate to actual wrong or misleading assessments. We conclude with some guidelines that we hope will help improve experimental evaluations of fuzz testing algorithms, making reported results more robust. CCS CONCEPTS• Security and privacy → Software and application security;A fuzz tester (or fuzzer) is a tool that iteratively and randomly generates inputs with which it tests a target program. Despite appearing "naive" when compared to more sophisticated tools involving SMT solvers, symbolic execution, and static analysis, fuzzers are surprisingly effective. For example, the popular fuzzer AFL has been used to find hundreds of bugs in popular programs [1]. Comparing AFL head-to-head with the symbolic executor angr, AFL found 76% more bugs (68 vs. 16) in the same corpus over a 24-hour period [50]. The success of fuzzers has made them a popular topic of research.

show abstract

Section: Recent Advances In Fuzzingmentioning

confidence: 99%

Section: Platform and Configurationmentioning

confidence: 99%

Evaluating Fuzz Testing

Klees

Ruef

Cooper

et al. 2018

Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

423

278

View full text Add to dashboard Cite

show abstract

“…Driller [58] combines fuzzing and concolic execution to discover deep bugs. Kargén and Shahmehri [37] perform mutations on the machine code of the generating programs instead of directly on a test input in order to leverage the information about the input format encoded in the generating programs. In summary, these fuzzing techniques target programs that process compact or unstructured inputs, which become less effective for programs that process structured inputs.…”

Section: Related Workmentioning

confidence: 99%

Superion: Grammar-Aware Greybox Fuzzing

Wang

Chen

Wei

et al. 2019

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

173

View full text Add to dashboard Cite

In recent years, coverage-based greybox fuzzing has proven itself to be one of the most effective techniques for finding security bugs in practice. Particularly, American Fuzzy Lop (AFL for short) is deemed to be a great success in fuzzing relatively simple test inputs. Unfortunately, when it meets structured test inputs such as XML and JavaScript, those grammar-blind trimming and mutation strategies in AFL hinder the effectiveness and efficiency.To this end, we propose a grammar-aware coverage-based greybox fuzzing approach to fuzz programs that process structured inputs. Given the grammar (which is often publicly available) of test inputs, we introduce a grammar-aware trimming strategy to trim test inputs at the tree level using the abstract syntax trees (ASTs) of parsed test inputs. Further, we introduce two grammar-aware mutation strategies (i.e., enhanced dictionary-based mutation and tree-based mutation). Specifically, tree-based mutation works via replacing subtrees using the ASTs of parsed test inputs. Equipped with grammar-awareness, our approach can carry the fuzzing exploration into width and depth.We implemented our approach as an extension to AFL, named Superion; and evaluated the effectiveness of Superion on real-life large-scale programs (a XML engine libplist and three JavaScript engines WebKit, Jerryscript and ChakraCore). Our results have demonstrated that Superion can improve the code coverage (i.e., 16.7% and 8.8% in line and function coverage) and bug-finding capability (i.e., 31 new bugs, among which we discovered 21 new vulnerabilities with 16 CVEs assigned and 3.2K USD bug bounty rewards received) over AFL and jsfunfuzz. We also demonstrated the effectiveness of our grammar-aware trimming and mutation.

show abstract

“…It even requires huge extra work for dynamic methods to generate test cases in order to cover the target function. Unfortunately, code coverage is still an issue for dynamic analysis of binaries [24].…”

Section: A Motivating Examplementioning

confidence: 99%

BinMatch: A Semantics-Based Hybrid Approach on Binary Code Clone Analysis

Zhang

et al. 2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Binary code clone analysis is an important technique which has a wide range of applications in software engineering (e.g., plagiarism detection, bug detection). The main challenge of the topic lies in the semantics-equivalent code transformation (e.g., optimization, obfuscation) which would alter representations of binary code tremendously. Another challenge is the trade-off between detection accuracy and coverage. Unfortunately, existing techniques still rely on semantics-less code features which are susceptible to the code transformation. Besides, they adopt merely either a static or a dynamic approach to detect binary code clones, which cannot achieve high accuracy and coverage simultaneously.In this paper, we propose a semantics-based hybrid approach to detect binary clone functions. We execute a template binary function with its test cases, and emulate the execution of every target function for clone comparison with the runtime information migrated from that template function. The semantic signatures are extracted during the execution of the template function and emulation of the target function. Lastly, a similarity score is calculated from their signatures to measure their likeness. We implement the approach in a prototype system designated as BINMATCH which analyzes IA-32 binary code on the Linux platform. We evaluate BINMATCH with eight real-world projects compiled with different compilation configurations and commonly-used obfuscation methods, totally performing over 100 million pairs of function comparison. The experimental results show that BINMATCH is robust to the semantics-equivalent code transformation. Besides, it not only covers all target functions for clone analysis, but also improves the detection accuracy comparing to the state-of-the-art solutions.

show abstract

Turning programs against each other: high coverage fuzz-testing using binary-code mutation and dynamic slicing

Cited by 48 publications

References 16 publications

Evaluating Fuzz Testing

Evaluating Fuzz Testing

Superion: Grammar-Aware Greybox Fuzzing

BinMatch: A Semantics-Based Hybrid Approach on Binary Code Clone Analysis

Contact Info

Product

Resources

About