Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique

Papadakis, Mike; Jia, Yunyi; Harman, Mark; Traon, Yves Le

doi:10.1109/icse.2015.103

Cited by 124 publications

(164 citation statements)

References 56 publications

Supporting

Mentioning

159

Contrasting

Order By: Relevance

“…The results reported by Papadakis et al [23] with respect to efficiency suggest that TCE is reasonably fast even for the most aggressive optimisation option: there is a trade-off between the compilation time and the effectiveness of the optimisation option, and the time required for equivalent and duplicate mutant detection is quite small when compared to the compilation time.…”

Section: Trivial Compiler Equivalencementioning

confidence: 74%

“…Papadakis et al [23] do note that compiler settings may influence equivalence detection, but they suggest that this should only cause false negatives. They claim that their technique (correctly implemented) cannot classify a nonequivalent mutant as equivalent.…”

Section:  Test Execution: (From "Mutants After Tce")mentioning

confidence: 99%

“…This is not, however, guaranteed. For detecting equivalent mutants, the experiments by Papadakis et al [23] show no clear winner among gcc's optimisation options. For detecting duplicate mutants, Papadakis et al observe that the best options are -O2 and -O3.…”

Section: Trivial Compiler Equivalencementioning

confidence: 99%

“…Mutant classification strategies analysing coverage impact of mutations [21], [22] have been used in a study to mitigate the effects of equivalence. Papadakis et al [23] also proposed a technique based on compiler optimisations (TCE) to automatically detect equivalent mutants. In their study, TCE was able to remove 30% of all existing equivalent mutants on average in 18 benchmark programs, and 7% (equivalent) and 21% (duplicate) of all mutants on 6 large opensource programs.…”

Section: Improving the Efficiency Of Mutation Testingmentioning

confidence: 99%

“… The compilation process when applying an optimisation setting (in the experiments by Papadakis et al [23] this was almost five times higher for -O3 than using no optimisation option);  The equivalence detection process (one comparison between the binary of each mutant and the original program); and  The duplicate mutant detection (each mutant is compared with the other mutants generated in the same function). The results reported by Papadakis et al [23] with respect to efficiency suggest that TCE is reasonably fast even for the most aggressive optimisation option: there is a trade-off between the compilation time and the effectiveness of the optimisation option, and the time required for equivalent and duplicate mutant detection is quite small when compared to the compilation time.…”

Section: Trivial Compiler Equivalencementioning

confidence: 99%

See 4 more Smart Citations

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

Section: Trivial Compiler Equivalencementioning

confidence: 74%

Section:  Test Execution: (From "Mutants After Tce")mentioning

confidence: 99%

Section: Trivial Compiler Equivalencementioning

confidence: 99%

Section: Improving the Efficiency Of Mutation Testingmentioning

confidence: 99%

Section: Trivial Compiler Equivalencementioning

confidence: 99%

See 3 more Smart Citations

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

Dissimilarity‐based test case prioritization through data fusion

Huang

Towey

et al. 2022

Softw Pract Exp

View full text Add to dashboard Cite

Test case prioritization (TCP) aims at scheduling test case execution so that more important test cases are executed as early as possible. Many TCP techniques have been proposed, according to different concepts and principles, with dissimilarity-based TCP (DTCP) prioritizing tests based on the concept of test case dissimilarity: DTCP chooses the next test case from a set of candidates such that the chosen test case is farther away from previously selected test cases than the other candidates. DTCP techniques typically only use one aspect/granularity of the information or features from test cases to support the prioritization process. In this article, we adopt the concept of data fusion to propose a new family of DTCP techniques, data-fusion-driven DTCP (DDTCP), which attempts to use different information granularities for prioritizing test cases by dissimilarity. We performed an empirical study involving 30 versions of five subject programs, investigating the testing effectiveness and efficiency by comparing DDTCP against DTCP techniques that use a dissimilarity granularity. The experimental results show that not only does DDTCP have better fault-detection rates than single-granularity DTCP techniques, but it also appears to only incur similar prioritization costs. The results also show that DDTCP remains robust over multiple system releases.

show abstract

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s, and remarkable advances have been achieved in its concepts, theory, technology, and empirical evidence.While the most influential realisations have been summarised by existing literature reviews, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 191 papers published between 1981 and 2015. In particular, we analysed in which quality assurance processes mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the inherent core problems of mutation testing, ie, the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped.Based on our observations, we made 9 recommendations for future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner. KEYWORDSapplication, mutation testing, systematic literature review 1 INTRODUCTION Mutation testing is defined by Jia and Harman [1] as a fault-based testing technique, which provides a testing criterion called the mutation adequacy score. This score can be used to measure the effectiveness of a test set in terms of its ability to detect faults [1]. The principle of mutation testing is to introduce syntactic changes into the original program to generate faulty versions (called mutants) according to well-defined rules (mutation operators) [2]. Mutation testing originated in the 1970s with works from Lipton [3], DeMillo et al. [4], and Hamlet [5] and has been a very active research field over the last few decades. The activeness of the field is in part evidenced by the extensive survey of more than 390 papers on mutation testing that Jia and Harman published in 2011 [1]. Jia and Harman's survey highlights the research achievements that have been made over the years, including the development of tools for a variety of languages and empirical studies performed [1]. Additionally, they highlight some of the actual and inherent problems of mutation testing, among others: (1) the high computational cost caused by generating and executing the numerous mutants and (2) the tremendous time-consuming human investigation required by the test oracle problem and equivalent mutant detection.While existing surveys (eg [1,2,6]) provide us with a great overview of the most influential realisations in research, we lack insight into how mutation testing is actually applied. Specifically, we are interested in analysing ...

show abstract

Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique

Cited by 124 publications

References 56 publications

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Dissimilarity‐based test case prioritization through data fusion

A systematic literature review of how mutation testing supports quality assurance processes

Contact Info

Product

Resources

About