Continuously reasoning about programs using differential Bayesian inference

Abstract

Programs often evolve by continuously integrating changes from multiple programmers. The effective adoption of program analysis tools in this continuous integration setting is hindered by the need to only report alarms relevant to a particular program change. We present a probabilistic framework, Drake, to apply program analyses to continuously evolving programs. Drake is applicable to a broad range of analyses that are based on deductive reasoning. The key insight underlying Drake is to compute a graph that concisely and precisely captures differences between the derivations of alarms produced by the given analysis on the program before and after the change. Performing Bayesian inference on the graph thereby enables to rank alarms by likelihood of relevance to the change. We evaluate Drake using SparrowÐa static analyzer that targets buffer-overrun, format-string, and integer-overflow errorsÐon a suite of ten widely-used C programs each comprising 13kś112k lines of code. Drake enables to discover all true bugs by inspecting only 30 alarms per benchmark on average, compared to 85 (3× more) alarms by the same ranking approach in batch mode, and 118 (4× more) alarms by a differential approach based on syntactic masking of alarms which also misses 4 of the 26 bugs overall. CCS Concepts • Software and its engineering → Automated static analysis; Software evolution; • Mathematics of computing → Bayesian networks.

Typilus: neural type hints

“…Work also exists in a variety of program analysis domains. Si et al [55] learn to predict loop invariants, Mangal et al [47] and Heo et al [33] use machine learning methods to filter false positives from program analyses by learning from user feedback. Chibotaru et al [17] use weakly supervised learning to learn taint specifications and DeFreez et al [23] mine API call sequence specifications.…”

“…Finally, graphs, which Typilus employs, encode a variety of complex relationships among the elements of code. Examples of include the early works of Kremenek et al [40] and Raychev et al [51] and variations of GNNs [5,20,33].…”

Survey of Approaches for Postprocessing of Static Analysis Alarms

“…• Liang et al [107] have introduced a novel Expressive Defect Pattern Specification Notation (EDPSN) to define a resource-leak defect pattern more precisely, and have used it to prioritize resource leaks. • Heo et al [68] have proposed a technique to rank alarms generated on evolving code. They compute a graph that concisely and precisely captures differences between the derivations of alarms generated before and after the change.…”

Learning to Represent Programs with Property Signatures

“…In this paper, we demonstrate the utility of property signatures for program synthesis, using them to perform a type of premise selection as in Balog et al (2016). More broadly, however, we envision that property signatures could be useful across a broad range of problems, including algorithm induction (Devlin et al, 2017), improving code readability (Allamanis et al, 2014), and program analysis (Heo et al, 2019).…”