Predicting mutation score using source code and test suite metrics

Jalbert,; Bradbury, Jeremy S.

doi:10.1109/raise.2012.6227969

Cited by 19 publications

(17 citation statements)

References 44 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, PMT opens a new dimension in mutation testing which does not require mutant execution, and has been shown to be more efficient than state-of-the-art techniques that embody various existing optimizations. Note that Jalbert [97] also applied machine learning to mutation testing, while their technique only classifies the mutation score of a source code unit into three categories: low, medium, and high.…”

Section: Mutation Testingmentioning

confidence: 99%

Predictive Mutation Testing

Zhang

Harman

et al. 2019

IIEEE Trans. Software Eng.

113

110

View full text Add to dashboard Cite

Abstract-Test suites play a key role in ensuring software quality. A good test suite may detect more faults than a poor-quality one. Mutation testing is a powerful methodology for evaluating the fault-detection ability of test suites. In mutation testing, a large number of mutants may be generated and need to be executed against the test suite under evaluation to check how many mutants the test suite is able to detect, as well as the kind of mutants that the current test suite fails to detect. Consequently, although highly effective, mutation testing is widely recognized to be also computationally expensive, inhibiting wider uptake. To alleviate this efficiency concern, we propose Predictive Mutation Testing (PMT): the first approach to predicting mutation testing results without executing mutants. In particular, PMT constructs a classification model, based on a series of features related to mutants and tests, and uses the model to predict whether a mutant would be killed or remain alive without executing it. PMT has been evaluated on 163 real-world projects under two application scenarios (cross-version and cross-project). The experimental results demonstrate that PMT improves the efficiency of mutation testing by up to 151.4X while incurring only a small accuracy loss. It achieves above 0.80 AUC values for the majority of projects, indicating a good tradeoff between the efficiency and effectiveness of predictive mutation testing. Also, PMT is shown to perform well on different tools and tests, be robust in the presence of imbalanced data, and have high predictability (over 60% confidence) when predicting the execution results of the majority of mutants.

show abstract

Section: Mutation Testingmentioning

confidence: 99%

Predictive Mutation Testing

Zhang

Harman

et al. 2019

IIEEE Trans. Software Eng.

113

110

View full text Add to dashboard Cite

show abstract

“…The presence of such mutants can help testers to improve the quality of their test suite. For mature test suites, we expect the number of non-equivalent surviving mutants to be low [30]. We further compare MUTANDIS against random mutation testing to evaluate the effect of our approach on the stubbornness of the generated mutants.…”

Section: Methodsmentioning

confidence: 99%

Guided Mutation Testing for JavaScript Web Applications

Mirshokraie

Mesbah

Pattabiraman

2015

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Mutation testing is an effective test adequacy assessment technique. However, there is a high computational cost in executing the test suite against a potentially large pool of generated mutants. Moreover, there is much effort involved in filtering out equivalent mutants. Prior work has mainly focused on detecting equivalent mutants after the mutation generation phase, which is computationally expensive and has limited efficiency. We propose an algorithm to select variables and branches for mutation as well as a metric, called F unctionRank, to rank functions according to their relative importance from the application's behaviour point of view. We present a technique that leverages static and dynamic analysis to guide the mutation generation process towards parts of the code that are more likely to influence the program's output. Further, we focus on the JavaScript language, and propose a set of mutation operators that are specific to web applications. We implement our approach in a tool called MUTANDIS. The results of our empirical evaluation show that (1) more than 93% of generated mutants are non-equivalent, and (2) more than 75% of the surviving non-equivalent mutants are in the top 30% of the ranked functions.

show abstract

“…Namin et al [42] used linear models to predict the overall mutation score, and Jalbert et al [22] also predicted that score using machine learning models. However, both did not perform predictions on individual methods.…”

Section: Related Workmentioning

confidence: 99%

Is the Stack Distance Between Test Case and Method Correlated With Test Effectiveness?

Niedermayr

Wagner

2019

Proceedings of the Evaluation and Assessment on Software Engineering

View full text Add to dashboard Cite

Mutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that. CCS CONCEPTS• Software and its engineering → Software testing and debugging;KEYWORDS software testing • test effectiveness • test metrics • minimal stack distance • mutation test prediction

show abstract

Predicting mutation score using source code and test suite metrics

Cited by 19 publications

References 44 publications

Predictive Mutation Testing

Predictive Mutation Testing

Guided Mutation Testing for JavaScript Web Applications

Is the Stack Distance Between Test Case and Method Correlated With Test Effectiveness?

Contact Info

Product

Resources

About