Combining static analysis error traces with dynamic symbolic execution (experience paper)

Abstract

This paper reports on our experience implementing a technique for sifting through static analysis reports using dynamic symbolic execution. Our insight is that if a static analysis tool produces a partial trace through the program under analysis, annotated with conditions that the analyser believes are important for the bug to trigger, then a dynamic symbolic execution tool may be able to exploit the trace by (a) guiding the search heuristically so that paths that follow the trace most closely are prioritised for exploration, and (b) pruning the search using the conditions associated with each step of the trace. This may allow the bug to be quickly confirmed using dynamic symbolic execution, if it turns out to be a true positive, yielding an input that triggers the bug.To experiment with this approach, we have implemented the idea in a tool chain that allows the popular open-source static analysis tools Clang Static Analyzer (CSA) and Infer to be combined with the popular open-source dynamic symbolic execution engine KLEE. Our findings highlight two interesting negative results. First, while fault injection experiments show the promise of our technique, they also reveal that the traces provided by static analysis tools are not that useful in guiding search. Second, we have systematically applied CSA and Infer to a large corpus of real-world applications that are suitable for analysis with KLEE, and find that the static analysers are rarely able to find non-trivial true positive bugs for this set of applications.We believe our case study can inform static analysis and dynamic symbolic execution tool developers as to where improvements may be necessary, and serve as a call to arms for researchers interested in combining symbolic execution and static analysis to identify more suitable benchmark suites for evaluation of research ideas. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

Uncovering Hidden Dependencies: Constructing Intelligible Path Witnesses using Dataflow Analyses

“…The authors in [13] discuss combining the Clang Static Analyzer with the dynamic symblic analyzer KLEE to refine the analysis. They highlight that traces provided by Clang are not that useful, and that Clang struggles to find non-trivial true positive.…”

Vulnerability verification with the assistance of directed symbolic execution

“…In this paper, we report our experiences investigating the functionality of directed symbolic execution in vulnerability verification scenario and designing a new heuristic search strategy called Dynamic Critical Constraint Modification (DCCM) search which utilize critical constraint to modify other forward symbolic execution strategy to reach the target program point quickly. As Busse F [7] concludes in their research, while fault injection experiments show the promise of guiding search by static report traces, they also reveal that those traces are not that useful in guiding search, even the same as just giving the vulnerability location as auxiliary. As a result, vulnerability verification problem serves as a line reachability problem, that is, given only a target point and a vulnerability type, generating test cases that can reproduce the vulnerability at the program point.…”