Experience report: How do techniques, programs, and tests impact automated program repair?

Abstract

Automated program repair can save tremendous manual efforts in software debugging. Therefore, a huge body of research efforts have been dedicated to design and implement automated program repair techniques. Among the existing program repair techniques, genetic-programming-based techniques have shown promising results. Recently, researchers found that random-search-based and adaptive program repair techniques can also produce effective results. In this work, we performed an extensive study for four program repair techniques, including genetic-programming-based, random-search-based, brute-forcebased and adaptive program repair techniques. Due to the extremely large time cost of the studied techniques, the study was performed on 153 bugs from 9 small to medium sized programs. In the study, we further investigated the impacts of different programs and test suites on effectiveness and efficiency of program repair techniques. We found that techniques that work well with small programs become too costly or ineffective when applied to medium sized programs. We also computed the false positive rates and discussed the ratio of the explored search space to the whole search space for each studied technique. Surprisingly, all the studied techniques except the random-searchbased technique are consistent with the 80/20 rule, i.e., about 80% of successful patches are found within the first 20% of search space.

Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset

“…The advantages of using Defects4J for a automatic repair experiment are: (realism) it contains real bugs (as opposed to seeded bugs as in Nguyen et al (2013); Kong et al (2015)); (scale) bugs are in large software (as opposed to bugs in student programs as in Smith et al (2015)); (novelty) nobody has ever studied Defects4J for repair.…”

Automated repair of feature interaction failures in automated driving systems

“…Several techniques use random search (e.g., RSRepair [37] and SCRepair [19]) to automate program repair. Although RSRepair [37] indicates that random search performs better than GenProg in terms of the number of iterations required for repair, a recent study [24] showed that GenProg performs better than RSRepair when applied to subjects different from those included in the original dataset of GenProg. We included two baselines based on random search in our evaluation (RS and RS-MM in Section 4).…”

Discovering common bug‐fix patterns: A large‐scale observational study

“…There are several advantages of using Defects4J for a study. Among them, we can highlight: Realism : It contains real bugs (as opposed to seeded bugs as in Nguyen et al and Kong et al). Scale : It contains bugs that reside in large software projects (as opposed to bugs in student programs as in Smith et al). Isolated Bugs : A fundamental challenge when collecting bugs is deciding what constitutes a bug, and what does not . When interacting with version control systems, developers frequently group separate changes into a single commit .…”