Commit-Aware Mutation Testing

Ma, Wei; Laurent, Thomas; Ojdanić, Miloš; Chekam, Thierry Titcheu; Ventresque, Anthony; Papadakis, Mike

doi:10.1109/icsme46990.2020.00045

Cited by 13 publications

(32 citation statements)

References 46 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, mutants interact with program changes when the post-commit mutant version (includes both the changes and the mutant) behaves differently from a) the related pre-commit mutant version and b) the post-commit non-mutated version. These conditions establish that changes and mutants interact [16] . Condition a) establishes that the behaviour differences are caused by the presence and absence of the committed changes and condition b) that the behaviour differences are caused by the presence and absence of the mutants.…”

Section: Commit-relevant Mutantsmentioning

confidence: 89%

“…Moreover, the most severe regression issues are due to unforeseen interactions between the changed code and the rest of the program [14], [15] . Therefore, we aim at using mutation testing using the so-called Commit-relevant mutants [16].…”

Section: B Change-aware Regression Testingmentioning

confidence: 99%

“…The virtue of commit-relevant mutation testing, as described in the study of Ma et al [16] is the best-effort application of mutation testing. This gives the potential for improved fault revelation under the same (relatively low) user effort than using randomly sampled mutants, i.e., traditional mutation testing.…”

Section: Commit-relevant Mutantsmentioning

confidence: 99%

“…We aim at testing commits using commit-relevant mutants; the subset of mutants on the post-commit program version that has a behaviour relevance to the committed changes [16].…”

Section: Ap P R O a C Hmentioning

confidence: 99%

“…The whole code base of Coreutils comprises approximately 60,000 lines of C code5. To perform our study on commits we used the benchmark6 introduced by Ma et al [16] that is composed of two parts and includes Benchmark-1, a set of commits mined fromthe Coreutils' Github repository from year 2012 to 2019 and CoREBench [19] that has fault introducing commits.…”

Section: A Benchmarks Usedmentioning

confidence: 99%

See 4 more Smart Citations

MuDelta: Delta-Oriented Mutation Testing at Commit Time

Chekam

Papadakis

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

To effectively test program changes using mutation testing, one needs to use mutants that are relevant to the altered program behaviours. We introduce MuDelta, an approach that identifies commit-relevant mutants; mutants that affect and are affected by the changed program behaviours. Our approach uses machine learning applied on a combined scheme of graph and vector-based representations of static code features. Our results, from 50 commits in 21 Coreutils programs, demonstrate a strong prediction ability of our approach; yielding 0.80 (ROC) and 0.50 (PR-Curve) AUC values with 0.63 and 0.32 precision and recall values. These predictions are significantly higher than random guesses, 0.20 (PR-Curve) AUC, 0.21 and 0.21 precision and recall, and subsequently lead to strong relevant tests that kill 45% more relevant mutants than randomly sampled mutants (either sampled from those residing on the changed component(s) or from the changed lines). Our results also show that MuDelta selects mutants with 27% higher fault revealing ability in fault introducing commits. Taken together, our results corroborate the conclusion that commit-based mutation testing is suitable and promising for evolving software.

show abstract

Section: Commit-relevant Mutantsmentioning

confidence: 89%

Section: B Change-aware Regression Testingmentioning

confidence: 99%

Section: Commit-relevant Mutantsmentioning

confidence: 99%

“…We aim at testing commits using commit-relevant mutants; the subset of mutants on the post-commit program version that has a behaviour relevance to the committed changes [16].…”

Section: Ap P R O a C Hmentioning

confidence: 99%

Section: A Benchmarks Usedmentioning

confidence: 99%

See 3 more Smart Citations

MuDelta: Delta-Oriented Mutation Testing at Commit Time

Chekam

Papadakis

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

show abstract

Learning‐based mutant reduction using fine‐grained mutation operators

Kim

Hong

2021

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryFor mutation testing, the huge cost of running test suites on a large number of mutants has been a serious obstacle. To resolve this problem, we propose a learning‐based mutant reduction technique MuTrain. MuTrain uses cost‐considerate linear regression (i.e., CLARS) to learn a mutation model, which predicts the mutation score of a test suite based on the mutation testing results of a previous version of a target program. Then, MuTrain applies the mutation model for subsequent versions to predict mutation scores with significantly fewer mutants. For effective mutant reduction and accurate mutation score prediction, MuTrain uses fine‐grained mutation operators refined from the existing coarse‐grained mutation operators. The experiment results show that MuTrain reduces the number of mutants effectively (i.e., selecting only 1.6% of mutants). Moreover, MuTrain predicts mutation score far more accurately than the existing mutant reduction techniques and random mutant selection. We also found that MuTrain achieves much greater mutant reduction when it uses the fine‐grained mutation operators than the traditional coarse‐grained mutation operators (i.e., 1.6% vs. 14.6%).

show abstract