Boosting fuzzer efficiency: an information theoretic perspective

Abstract

In this paper, we take the fundamental perspective of fuzzing as a learning process. Suppose before fuzzing, we know nothing about the behaviors of a program P: What does it do? Executing the first test input, we learn how P behaves for this input. Executing the next input, we either observe the same or discover a new behavior. As such, each execution reveals łsome amountž of information about P's behaviors. A classic measure of information is Shannon's entropy. Measuring entropy allows us to quantify how much is learned from each generated test input about the behaviors of the program. Within a probabilistic model of fuzzing, we show how entropy also measures fuzzer efficiency. Specifically, it measures the general rate at which the fuzzer discovers new behaviors. Intuitively, efficient fuzzers maximize information. From this information theoretic perspective, we develop Entropic, an entropy-based power schedule for greybox fuzzing which assigns more energy to seeds that maximize information. We implemented Entropic into the popular greybox fuzzer LibFuzzer. Our experiments with more than 250 open-source programs (60 million LoC) demonstrate a substantially improved efficiency and confirm our hypothesis that an efficient fuzzer maximizes information. Entropic has been independently evaluated and invited for integration into main-line LibFuzzer. Entropic now runs on more than 25,000 machines fuzzing hundreds of security-critical software systems simultaneously and continuously. CCS CONCEPTS • Software and its engineering → Software testing.

Hashing Fuzzing: Introducing Input Diversity to Improve Crash Detection

“…Our results show that the fuzzer skills also improve in terms of crash detection and coverage for several cases when diversity is embedded within the generation process. This is similar to the results of Böhme et al [25] when they leverage entropy to increase the diversity of behaviours during fuzzing campaigns. Therefore, r-wise independent hash functions compensate the adversarial behaviour of fuzzers and solvers during the input generation process by improving the fuzzers diversity.…”

“…Since our comparison is based on performance, using different machines will affect the final results. The results might also be affected if the fuzzers add a new diversity strategy, as was recently the case for LibFuzzer, that now includes the "Entropic" extension, improving diversity during the queue selection process [25].…”

Fuzzing: A Survey for Roadmap

“…To mitigate this problem, fuzzers calculate itness based on execution states (e.g., code coverage) because execution states can provide fuzzing with more information. Most existing fuzzers compute itness based on code coverage, i.e., they intend to discover more code coverage [21,23,62,88,157,203]. Another reason for using code coverage is that larger code coverage indicates a higher possibility of bug discovery [130].…”

“…However, the essential objective of fuzzing is to discover new states, e.g., new code coverage, new crashes, or new bugs. This motivates Böhme et al [17,20,21] to formulate fuzzing processes as a species discovery problem [32,33,52]. In a nutshell, ecologists collect numerous samples from the wild, and the species in the samples may be abundant or rare.…”

icLibFuzzer: Isolated-context libFuzzer for Improving Fuzzer Comparability

“…Entropic [3] improves libFuzzer's seed scheduling algorithm. It quantifies the amount of information gained by each seed and chooses which seed to mutate first according to this value.…”