Mutation-Based Fault Localization for Real-World Multilingual Programs (T)

Hong, Shin; Lee, Byeongcheol; Kwak, Taehoon; Jeon, Yiru; Ko, Bongsuk; Kim, Yun‐Ho; Kim, Moonzoo

doi:10.1109/ase.2015.14

Cited by 52 publications

(28 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work of Hong et al. on MUSEUM and Moon et al. on MUSE provides a good summary of other work along these lines and is perhaps most similar to ours in assumptions.…”

Section: Related Worksupporting

confidence: 76%

“…A dash for a column means that localization did not rank any faulty statements or assigned all faulty statements suspiciousness of 0.0. The three rankings are our Repair localization, the MUSE localization and the MUSEUM localization. MUSEUM uses the same formula as MUSE but works better for multiple faults because it uses only one failure.…”

Section: Core Experimental Results: Compiler Triage and Debuggingmentioning

confidence: 99%

“…Preliminary investigation also shows improvement (for a shared mutation testing budget) over previous mutant‐based fault localization methods, for the same tests and a subset of 24 faults with known locations. Our Repair method, in particular, was the only localization approach to provide any perfect localizations of faults (it produced three) and provided three times as many very high‐quality (ranking the fault in top 5 locations) localizations as the next best method – six such localizations, compared with only two. For GCC wrong code faults, only our methods produced any very high‐quality fault localizations.…”

Section: Discussionmentioning

confidence: 99%

“…However, it is possible to compare Repair with other methods, using their full mutation and experimental set, to provide a more generalizable (and fair) comparison than the evaluation versus MUSE and MUSEUM over compiler faults. We obtained the data from two evaluations of mutant‐based fault localization methods: the set of faults used to evaluate MUSEUM and a set of CoREBench faults used in an evaluation of MUSE and Metallaxis‐FL by Chekal et al (we were unable to obtain the original MUSE data set, thus far). These data sets are not ideal for evaluating Repair : for all but 10 of the faults (all from the CoREBench set), there is only a single failing test case in the data.…”

Section: Repair Localization For Non‐compiler Faultsmentioning

confidence: 99%

“…We show that such a metric can improve, in a statistically significant way, the ability to distinguish tests based on which faults they trigger. Furthermore, fault localization based on the same metrics can successfully pinpoint the location of some compiler faults sufficiently to be likely to substantially aid debugging and, when applicable, performs very well for some faults studied in recent papers on mutant‐based fault localization .…”

Section: Introductionmentioning

confidence: 97%

See 4 more Smart Citations

Using mutants to help developers distinguish and debug (compiler) faults

Holmes

Groce

2020

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary Measuring the distance between two program executions is a fundamental problem in dynamic analysis of software and useful in many test generation and debugging algorithms. This paper proposes a metric for measuring distance between executions and specializes it to an important application: determining similarity of failing test cases for the purpose of automated fault identification and localization in debugging based on automatically generated compiler tests. The metric is based on a causal concept of distance where executions are similar to the degree that changes in the program itself, introduced by mutation, cause similar changes in the correctness of the executions. Specifically, if two failing test cases for a compiler can be made to pass by applying the same mutant, those two tests are more likely to be due to the same fault. We evaluate our metric using more than 50 faults and 2,800 test cases for two widely used real‐world compilers and demonstrate improvements over state‐of‐the‐art methods for fault identification and localization. A simple operator selection approach to reducing the number of mutants can reduce the cost of our approach by 70%, while producing a gain in fault identification accuracy. We additionally show that our approach, although devised for compilers, is applicable as a conservative fault localization algorithm for other types of programs and can help triage certain types of crashes found in fuzzing non‐compiler programs more effectively than a state‐of‐the‐art technique.

show abstract

“…The work of Hong et al. on MUSEUM and Moon et al. on MUSE provides a good summary of other work along these lines and is perhaps most similar to ours in assumptions.…”

Section: Related Worksupporting

confidence: 76%

Section: Core Experimental Results: Compiler Triage and Debuggingmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Repair Localization For Non‐compiler Faultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

See 3 more Smart Citations

Using mutants to help developers distinguish and debug (compiler) faults

Holmes

Groce

2020

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

Emerging Aspects of Software Fault Localization

Tse¹,

Gorce

et al. 2023

Handbook of Software Fault Localization

View full text Add to dashboard Cite

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

et al. 2019

View full text Add to dashboard Cite

Pure spectrum-based fault localization (SBFL) is a well-studied statistical debugging technique that only takes a set of test cases (some failing and some passing) and their code coverage as input and produces a ranked list of suspicious program elements to help the developer identify the location of a bug that causes a failed test case. Studies show that pure SBFL techniques produce good ranked lists for small programs. However, our previous study based on the iBugs benchmark that uses the AspectJ repository shows that, for realistic programs, the accuracy of the ranked list is not suitable for human developers. In this paper, we confirm this based on a combined empirical evaluation with the iBugs and the Defects4J benchmark. Our experiments show that, on average, at most ∼40%, ∼80%, and ∼90% of the bugs can be localized reliably within the first 10, 100, and 1000 ranked lines, respectively, in the Defects4J benchmark.To reliably localize 90% of the bugs with the best performing SBFL metric D * , ∼450 lines have to be inspected by the developer. For human developers, this remains unsuitable, although the results improve compared with the results for the AspectJ benchmark. Based on this study, we can clearly see the need to go beyond pure SBFL and take other information, such as information from the bug report or from version history of the code lines, into consideration.

show abstract

Mutation-Based Fault Localization for Real-World Multilingual Programs (T)

Cited by 52 publications

References 2 publications

Using mutants to help developers distinguish and debug (compiler) faults

Using mutants to help developers distinguish and debug (compiler) faults

Emerging Aspects of Software Fault Localization

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

Contact Info

Product

Resources

About