A machine learning approach for classification of equivalent mutants

Abstract

Mutation testing is a fault-based technique to test the quality of test suites by inducing artificial syntactic faults or mutants in a source program. However, some mutants have the same semantics as original program and cannot be detected by any test suite input known as equivalent mutants. Equivalent mutant problem (EMP) is undecidable as it requires manual human effort to identify a mutant as equivalent or killable. The constraint-based testing (CBT) theory suggests the use of mathematical constraints which can help reveal some equivalent mutants using mutant features. In this paper, we consider three metrics of CBT theory, ie, reachability, necessity, and sufficiency to extract feature constraints from mutant programs. Constraints are extracted using program dependency graphs. Other features such as degree of significance, semantic distance, and information entropy of mutants are also extracted to build a binary classification model. Machine learning algorithms such as Random Forest, GBT, and SVM are applied under two application scenarios (split-project and cross-project) on ten Java programs to predict equivalent mutants. The analysis of the study demonstrates that that the proposed techniques not only improves the efficiency of the equivalent mutant detection but also reduces the effort required to perform it with small accuracy loss.

Automatic Classification of Equivalent Mutants in Mutation Testing of Android Applications

“…The loss function of both training and validation is shown in Figure 9. The obtained accuracy of 94% outperforms the models proposed in [2,5,7]. Furthermore, our results show fewer False Negatives during testing and lower False Positives, which is significant as it implies that there is a decrease in the risk of labeling a mutant as equivalent when it is not.…”

“…There were equally lower False Positives, thereby indicating a decrease in the risk of labelling a mutant as equivalent to the strengthening of the test suites. Furthermore, to adequately evaluate our model, we achieved a 94% accuracy, higher than other models [2,5,17], which shows it predicted very few False Negatives during testing, see Figure 10. There were equally lower False Positives, thereby indicating a decrease in the risk of labelling a mutant as equivalent to the strengthening of the test suites.…”

“…Recently, a lot has been achieved in the area of research that effectively aids the mutation testing of android apps [1,5,12]. To adequately certify the test suites that these studies recommended, the mutation technique is used to determine the fault detection capabilities of these tools.…”

“…Another paper used deep CNN (DCNN) for the same classification but returned an accuracy of 60% with a great focus on incremental learning [19]. The work of [8] reduced the mutant testing effort using their MuAPK 2.0 tool through the removal of dead code mutants with an accuracy of 90%, while [5] used mathematical constraints in constraint-based testing (CBT) theory and Random Forest to achieve 80% accuracy in their approach for the classification of equivalent mutants. Furthermore, refs.…”

“…The SUT is mutated through the use of mutation operators to change some part of the source codes, thereby increasing the chances of the test suites giving two different outputs between the SUT and the Mutant program. If the mutants are killed, then we compute the mutation score [5], which is the basis for evaluating the test suites. The issue with this technique is the process involved in identifying the equivalent mutants within the mutant program after the mutants are killed because the mutation score computation relies heavily on the identification of these types of mutants.…”