Mutant Subsumption Graphs

Abstract-The question of which are the valuable mutants has received little attention in mutation testing literature. Naturally, the choice of mutants impacts the quality of the performed analysis and has the potential of changing the conclusions of empirical studies. To this end, we collect definitions related to mutant quality indicators and analyses their relations. We identify two classes of indicators, related to individual mutants (Fault Revealing, Subsuming, Hard-to-kill and Stubborn) and to mutant sets (disjoint/dominator and distinguished). We analyse a large set of mutants from 3,902 (real) faulty program versions, belonging to 40 fault classes, collected from an on-line programming contest. Our analysis categorises mutants as valuable, according to the studied quality indicators, profiles their types and examines the relations between them. Our results suggest that there is a large disagreement between the indicators and that the connection between mutant type, its quality and its ability to reveal faults is weak. Additionally, our paper reveal that the ability of mutants to uncover faults differs significantly across the different fault classes and that some mutant types are well linked (or completely disconnected) to specific fault classes.

show abstract

“…We consider as subsuming, all mutants that are in the leaf nodes of the mutant subsumption graphs [10], built based on the employed test suites.…”

Section: A Unit-based Mqismentioning

confidence: 99%

“…Surface mutants [12] are also approximated by a similar heuristic. The minimal/dominator mutants form the actual minimal subset, selected bases on a systematic procedure [10]. Higher Order The set-based indicators depend on the individual choice of the individual mutants and cannot be compared with the unitbased ones.…”

Section: B Set-based Mqismentioning

confidence: 99%

Mutant Quality Indicators

Papadakis

Chekam

Traon

2018

2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

View full text Add to dashboard Cite

show abstract

“…Kurtz et al defined the following three types of subsumption relations, i.e., true subsumption, dynamic subsumption and static subsumption, and built a graph model to represent the subsumption relation between mutants [45]. In addition, they indicated that the true subsumption is not computable, the dynamic subsumption which is an approximation to the true one can be detected by a specific test set, and the static subsumption which approaches the true one can be manually or automatically analyzed.…”

Section: Testability Transformation and Mutant Reductionmentioning

confidence: 99%

Mutant reduction based on dominance relation for weak mutation testing

Gong

Zhang

Yao

et al. 2017

Information and Software Technology

View full text Add to dashboard Cite

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. AbstractContext: As a fault-based testing technique, mutation testing is effective at evaluating the quality of existing test suites. However, a large number of mutants result in the high computational cost in mutation testing. As a result, mutant reduction is of great importance to improve the efficiency of mutation testing. Objective: We aim to reduce mutants for weak mutation testing based on the dominance relation between mutant branches. Method: In our method, a new program is formed by inserting mutant branches into the original program. By analyzing the dominance relation between mutant branches in the new program, the non-dominated one is obtained, and the mutant corresponding to the non-dominated mutant branch is the mutant after reduction. Results: The proposed method is applied to test ten benchmark programs and six classes from open-source projects.The experimental results show that our method reduces over 80% mutants on average, which greatly improves the efficiency of mutation testing. Conclusion: We conclude that dominance relation between mutant branches is very important and useful in reducing mutants for mutation testing.

show abstract

“…There has recent interest in mutant subsumption as a means of improving mutation effectiveness by removing subsumed mutants from consideration for first order mutation testing [24,27]. The idea of subsuming higher order mutation was first proposed by Jia and Harman [20,22].…”

Section: Related Workmentioning

confidence: 99%

Angels and monsters

Harman

Jia

Mateo

et al. 2014

Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

View full text Add to dashboard Cite

We study the simultaneous test effectiveness and efficiency improvement achievable by Strongly Subsuming Higher Order Mutants (SSHOMs), constructed from 15,792 first order mutants in four Java programs. Using SSHOMs in place of the first order mutants they subsume yielded a 35%-45% reduction in the number of mutants required, while simultaneously improving test efficiency by 15% and effectiveness by between 5.6% and 12%. Trivial first order faults often combine to form exceptionally non-trivial higher order faults; apparently innocuous angels can combine to breed monsters. Nevertheless, these same monsters can be recruited to improve automated test effectiveness and efficiency.

show abstract

Mutant Subsumption Graphs

Cited by 80 publications

References 38 publications

Mutant Quality Indicators

Mutant Quality Indicators

Mutant reduction based on dominance relation for weak mutation testing

Angels and monsters

Contact Info

Product

Resources

About