Automated Fixing of Programs with Contracts

Abstract

This paper describes AutoFix, an automatic debugging technique that can fix faults in general-purpose software. To provide high-quality fix suggestions and to enable automation of the whole debugging process, AutoFix relies on the presence of simple specification elements in the form of contracts (such as pre-and postconditions). Using contracts enhances the precision of dynamic analysis techniques for fault detection and localization, and for validating fixes. The only required user input to the AutoFix supporting tool is then a faulty program annotated with contracts; the tool produces a collection of validated fixes for the fault ranked according to an estimate of their suitability.In an extensive experimental evaluation, we applied AutoFix to over 200 faults in four code bases of different maturity and quality (of implementation and of contracts). AutoFix successfully fixed 42% of the faults, producing, in the majority of cases, corrections of quality comparable to those competent programmers would write; the used computational resources were modest, with an average time per fix below 20 minutes on commodity hardware. These figures compare favorably to the state of the art in automated program fixing, and demonstrate that the AutoFix approach is successfully applicable to reduce the debugging burden in real-world scenarios.

Contract-based program repair without the contracts

“…This gives a nuanced picture of the results, which must however be taken-as usual-with a grain of salt: different tools may focus on achieving a better ranking vs. correctly fixing more bugs, and we do not imply that there is one universal measure of effectiveness. Anyway, our evaluation is widely applicable-including to papers that may not detail this aspect-and is in line with what done in other evaluations [14], [17], [25], [28], [29].…”

“…We refrain from quantitatively comparing APR tools that target other programming languages-and thus were evaluated on different benchmarks [15]. Nevertheless, just to give an idea, Angelix [22] and Prophet [17] 6 achieve a precision of 35.7% and 42.9%, and a recall of 9.5% and 17%, on 105 bugs in the C GenProg benchmark [14]; AutoFix [25] 7 achieves a precision of 59.3% and a recall of 25% on 204 bugs from various Eiffel projects with contracts.…”

“…1 Purity analysis. One lesson that we can draw from the experience of contract-based APR [25] is that constructing a rich set of expressions that abstract the program state can help support more accurate fault localization and fix generation, and ultimately the construction of higher-quality "semantic" fixes that are less prone to overfitting. However, monitoring a rich set of expressions extracted from the program text does not work as well in languages such as Java as it does in languages that support contracts.…”

“…By also relying on contracts (specifications embedded in the program text) AutoFix [25], [26], [31] was the first generalpurpose APR technique to substantially increase the number of correct fixes-for 25% of 204 bugs in [25]. JAID generalizes AutoFix's state-based analysis to work on Java code without contracts, so as to improve the quality of the generated fixes without sacrificing applicability.…”

“…Even if the contracts used by AutoFix are far from being detailed, let alone complete, method specifications, they significantly help increase the fraction of correct fixes that can be generated [25]. Unfortunately, even such simple contracts are hardly ever available in the most widely used programming languages.…”