A mutation carol: Past, present and future

Offutt, Jeff

doi:10.1016/j.infsof.2011.03.007

Cited by 74 publications

(95 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome the equivalent mutant problem, there are mainly 3 categories classified by Madeyski et al [6]: (1) detecting equivalent mutants, such as Baldwin and Sayward [38] (using compiler optimisations), Hierons et al [39] (using program slicing), Martin and Xie [40] (through change-impact analysis), Ellims et al [41] (using running profile), and du Bousquet and Delaunay [42] (using model checker); (2) avoiding equivalent mutant generation, such as Mresa and Bottaci [31] (through selective mutation), Harman et al [43] (using program dependence analysis), and Adamopoulos et al [44] (using co-evolutionary search algorithm); (3) suggesting equivalent mutants, such as Bayesian learning [45], dynamic invariants analysis [46], and coverage change examination (eg [47]).…”

Section: Benefits and Limitationsmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s, and remarkable advances have been achieved in its concepts, theory, technology, and empirical evidence.While the most influential realisations have been summarised by existing literature reviews, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 191 papers published between 1981 and 2015. In particular, we analysed in which quality assurance processes mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the inherent core problems of mutation testing, ie, the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped.Based on our observations, we made 9 recommendations for future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner. KEYWORDSapplication, mutation testing, systematic literature review 1 INTRODUCTION Mutation testing is defined by Jia and Harman [1] as a fault-based testing technique, which provides a testing criterion called the mutation adequacy score. This score can be used to measure the effectiveness of a test set in terms of its ability to detect faults [1]. The principle of mutation testing is to introduce syntactic changes into the original program to generate faulty versions (called mutants) according to well-defined rules (mutation operators) [2]. Mutation testing originated in the 1970s with works from Lipton [3], DeMillo et al. [4], and Hamlet [5] and has been a very active research field over the last few decades. The activeness of the field is in part evidenced by the extensive survey of more than 390 papers on mutation testing that Jia and Harman published in 2011 [1]. Jia and Harman's survey highlights the research achievements that have been made over the years, including the development of tools for a variety of languages and empirical studies performed [1]. Additionally, they highlight some of the actual and inherent problems of mutation testing, among others: (1) the high computational cost caused by generating and executing the numerous mutants and (2) the tremendous time-consuming human investigation required by the test oracle problem and equivalent mutant detection.While existing surveys (eg [1,2,6]) provide us with a great overview of the most influential realisations in research, we lack insight into how mutation testing is actually applied. Specifically, we are interested in analysing ...

show abstract

Section: Benefits and Limitationsmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…To minimise the random error, we carry out the process of the mutant selection 100 times for each project, and compare the error rate, i.e. the ratio of mistakenly predicated mutants in strong mutation, as shown in Equation 1. For this experiment, we used all test cases.…”

Section: B Experimental Setupmentioning

confidence: 99%

“…Mutation testing introduces small syntactic changes into the program to generate faulty versions (mutants) according to well-defined rules (mutation operators) [1]. Then the quality of a test suite can be qualified as the percentage of mutants it distinguishes from the original program (mutation score).…”

Section: Introductionmentioning

confidence: 99%

Speeding-Up Mutation Testing via Data Compression and State Infection

Zhu

Panichella

Zaidman

2017

2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

View full text Add to dashboard Cite

Abstract-Mutation testing is widely considered as a high-end test criterion due to the vast number of mutants it generates. Although many efforts have been made to reduce the computational cost of mutation testing, its scalability issue remains in practice. In this paper, we introduce a novel method to speed up mutation testing based on state infection information. In addition to filtering out uninfected test executions, we further select a subset of mutants and a subset of test cases to run leveraging data-compression techniques. In particular, we adopt Formal Concept Analysis (FCA) to group similar mutants together and then select test cases to cover these mutants. To evaluate our method, we conducted an experimental study on six open source Java projects. We used EvoSuite to automatically generate test cases and to collect mutation data. The initial results show that our method can reduce the execution time by 83.93% with only 0.257% loss in precision.

show abstract

“…Over the years, mutation testing has been remarkably studied and applied not only to programs from different programming paradigms such as Fortran programs [19,20], C programs [21], Java programs [22], and AspectJ programs [12,23], but also to specifications or models of programs, such as Finite State Machine [24], Petri Nets [25], and Security Policies [26]. Surveys and reviews on work in mutation testing can be found in [27][28][29].…”

Section: Mutation Testing (Traditional)mentioning

confidence: 99%

Towards Semantic Mutation Testing of Aspect-Oriented Programs

Ghani¹

2013

JSEA

View full text Add to dashboard Cite

Aspect-oriented programs have received much attention from software testing researchers. Various testing techniques and approaches have been proposed to tackle issues and challenges when testing aspect-oriented programs including traditional mutation testing. In traditional mutation testing of aspect-oriented programs, mutants are generated by making small changes to the syntax of the aspect-oriented language. Recently, a new approach known as semantic mutation testing has been proposed. This approach mutates the semantics of the language in which the program is written. The mutants generated misunderstandings of the language which are different classes of faults. Aspect-oriented programming presents itself with different properties that can be further explored with respect to semantic mutation testing. This paper describes various possible scenarios that semantic mutation testing strategy might have particular value in testing aspect-oriented programs.

show abstract

A mutation carol: Past, present and future

Cited by 74 publications

References 85 publications

A systematic literature review of how mutation testing supports quality assurance processes

A systematic literature review of how mutation testing supports quality assurance processes

Speeding-Up Mutation Testing via Data Compression and State Infection

Towards Semantic Mutation Testing of Aspect-Oriented Programs

Contact Info

Product

Resources

About