GiGAn

Abstract

The current trend in mutation testing is to reduce the great testing effort that it involves, but it should be based on wellstudied cost reduction techniques. Evolutionary Mutation Testing (EMT) aims at generating a reduced set of mutants by means of an evolutionary algorithm, which searches for potentially equivalent and difficult to kill mutants to help improve the test suite. However, there is little evidence of its applicability to other contexts beyond WS-BPEL compositions. This study explores its performance when applied to C++ object-oriented programs thanks to a newly developed system, GiGAn. The conducted experiments reveal that EMT shows stable behavior in all the case studies, where the best results are obtained when a low percentage of the mutants is generated. They also support previous studies of EMT when compared to random mutant selection, reinforcing its use for the goal of improving the fault detection capability of the test suite. CCS Concepts•Software and its engineering → Software testing and debugging; Search-based software engineering;

Search-based mutant selection for efficient test suite improvement: Evaluation and results

“…They developed a genetic algorithm [8] in the tool GAmera [18] and applied the algorithm to test three WS-BPEL compositions. The same technique was later implemented for C++ code with the development of the tool GiGAn [6]. Our work supports the findings pointed out by Domínguez-Jiménez et al [16] regarding the ability of the technique to find strong mutants, but we go a step beyond by measuring the improvement that can be achieved thanks to the mutants selected in eight programs of varying nature.…”

“…The experiments conducted in this paper go a step beyond than previous experiments -where the ability of the technique to find strong mutants was evaluated-by assessing in depth a new methodology that allows us to estimate the extent to which EMT could help improve the test suite. This work connects and extends the results of two previous papers [6,7], providing a comprehensive picture regarding the behavior of EMT, descriptive examples on how to implement the proposed methodology, statistical significance of the results and a discussion comparing the old and the new methodology and the results of EMT, random selection and selective mutation. The paper also includes a list of lessons learned that can be useful for researchers in this field in a foreseeable future.…”

“…Namely, we analyzed eight case studies with respect to the set of class mutation operators implemented in MuCPP. Further details about GiGAn can be found in a previous paper [6]. The parameter values used for the genetic algorithm (see Sections 3.1 and 3.3) were those recommended in the literature [16]:…”