Predicting the fault revelation utility of mutants

Chekam, Thierry Titcheu; Papadakis, Mike; Bissyandé, Tegawendé F.; Traon, Yves Le

doi:10.1145/3183440.3195031

Cited by 5 publications

(4 citation statements)

References 6 publications

(4 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is because the fault‐detecting test cases are not known at the time of prioritization, and the thickness of edges in an MDG cannot be determined without the fault detection information. Fortunately, recent studies show that it is possible to predict and prioritize mutants that are more likely to be killed by fault‐detecting test cases than the others []. Such studies will help to extend the use cases of an MDG.…”

Section: Discussionmentioning

confidence: 99%

Empirical evaluation of mutation‐based test case prioritization techniques

Shin

Yoo

Papadakis

et al. 2018

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Summary In this paper, we propose a new test case prioritization technique that combines both mutation‐based and diversity‐aware approaches. The diversity‐aware mutation‐based technique relies on the notion of mutant distinguishment, which aims to distinguish one mutant's behaviour from another, rather than from the original program. The relative cost and effectiveness of the mutation‐based prioritization techniques (i.e., using both the traditional mutant kill and the proposed mutant distinguishment) are empirically investigated with 352 real faults and 553,477 developer‐written test cases. The empirical evaluation considers both the traditional and the diversity‐aware mutation criteria in various settings: single‐objective greedy, hybrid, and multi‐objective optimization. The results show that there is no single dominant technique across all the studied faults. To this end, the reason why each one of the mutation‐based prioritization criteria performs poorly is discussed, using a graphical model called Mutant Distinguishment Graph that demonstrates the distribution of the fault‐detecting test cases with respect to mutant kills and distinguishment. © 2018 John Wiley & Sons, Ltd.

show abstract

Section: Discussionmentioning

confidence: 99%

Empirical evaluation of mutation‐based test case prioritization techniques

Shin

Yoo

Papadakis

et al. 2018

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

show abstract

“…Codeflaws: Papadakis et al [28] collect and analyze mutant quality indicators based on Codeflaws. Chekam et al [3] propose a new perspective to tackle the fault revelation mutant selection and evaluate their work on Codeflaws.…”

Section: Threats To Validitymentioning

confidence: 99%

A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark

Yang

Martínez

Durieux

et al. 2019

2019 IEEE 1st International Workshop on Intelligent Bug Fixing (IBF)

View full text Add to dashboard Cite

Automatic program repair papers tend to repeatedly use the same benchmarks. This poses a threat to the external validity of the findings of the program repair research community. In this paper, we perform an automatic repair experiment on a benchmark called QuixBugs that has never been studied in the context of program repair. In this study, we report on the characteristics of QuixBugs, and study five repair systems, Arja, Astor, Nopol, NPEfix and RSRepair, which are representatives of generate-and-validate repair techniques and synthesis repair techniques. We propose three patch correctness assessment techniques to comprehensively study overfitting and incorrect patches. Our key results are: 1) 15 / 40 buggy programs in the QuixBugs can be repaired with a test-suite adequate patch; 2) a total of 64 plausible patches for those 15 buggy programs in the QuixBugs are present in the search space of the considered tools; 3) the three patch assessment techniques discard in total 33 / 64 patches that are overfitting. This sets a baseline for future research of automatic repair on QuixBugs. Our experiment also highlights the major properties and challenges of how to perform automated correctness assessment of program repair patches. All experimental results are publicly available on Github in order to facilitate future research on automatic program repair.

show abstract

“…These mutants are the ones that are linked with fault revelation. Thus, one should cover only the mutants that are most likely to be linked with frequently occuring (real) faults [7].…”

Section: A Unit-based Mqismentioning

confidence: 99%

“…A related, newly suggested indicator is that quality mutants are those that guide testers towards revealing real faults [7]. The underlying idea is that good mutants should lead to test cases that reveal frequent real faults.…”

Section: Introductionmentioning

confidence: 99%

Mutant Quality Indicators

Papadakis

Chekam

Traon

2018

2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

Self Cite

View full text Add to dashboard Cite

Abstract-The question of which are the valuable mutants has received little attention in mutation testing literature. Naturally, the choice of mutants impacts the quality of the performed analysis and has the potential of changing the conclusions of empirical studies. To this end, we collect definitions related to mutant quality indicators and analyses their relations. We identify two classes of indicators, related to individual mutants (Fault Revealing, Subsuming, Hard-to-kill and Stubborn) and to mutant sets (disjoint/dominator and distinguished). We analyse a large set of mutants from 3,902 (real) faulty program versions, belonging to 40 fault classes, collected from an on-line programming contest. Our analysis categorises mutants as valuable, according to the studied quality indicators, profiles their types and examines the relations between them. Our results suggest that there is a large disagreement between the indicators and that the connection between mutant type, its quality and its ability to reveal faults is weak. Additionally, our paper reveal that the ability of mutants to uncover faults differs significantly across the different fault classes and that some mutant types are well linked (or completely disconnected) to specific fault classes.

show abstract

Predicting the fault revelation utility of mutants

Cited by 5 publications

References 6 publications

Empirical evaluation of mutation‐based test case prioritization techniques

Empirical evaluation of mutation‐based test case prioritization techniques

A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark

Mutant Quality Indicators

Contact Info

Product

Resources

About