Selective mutation testing for concurrent code

Gligoric, Milos; Zhang, Lingming; Pereira, Cristiano; Pokam, Gilles

doi:10.1145/2483760.2483773

Cited by 53 publications

(54 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various methodologies for reducing the cost of mutation testing have been proposed. One widely used methodology is selective mutation testing [3]- [7], [9], [20], [21], which only generates and executes a subset of mutants for mutation testing. Ideally, the selected subset of mutants should be representative of the entire set of mutants.…”

Section: Introductionmentioning

confidence: 99%

“…The most widely studied approach for selective mutation testing is operator-based mutant selection [3]- [5], [7], [9], [20], which only generates mutants using a subset of mutation operators; the selected subset of mutation operators is required to be effective, i.e., if a test suite kills all the non-equivalent mutants generated by the selected set of operators (i.e., the test suite is adequate for selected mutants), then the test suite should kill (almost) all the non-equivalent mutants generated by all mutation operators. Further, selected operators should lead to high savings; the savings is usually calculated as the ratio of non-selected mutants over all the mutants.…”

Section: Introductionmentioning

confidence: 99%

“…Further, selected operators should lead to high savings; the savings is usually calculated as the ratio of non-selected mutants over all the mutants. Researchers also evaluated a simple approach of random mutant selection [7], [9], [22], and a recent study [7] reported that random selection is as effective as operator-based mutant selection when random selection selects the same number of mutants from all mutants as the operator-based selection selects.…”

Section: Introductionmentioning

confidence: 99%

“…• Sampling mutation: We investigate a simple idea, which we call sampling mutation, to reduce the number of mutants generated by operator-based mutant selection: sampling mutation randomly selects from the set of mutants generated by operator-based mutant selection (rather than from the set of mutants generated by all operators [7], [9], [22]); we call the process of obtaining the mutants sampling, the percentage of randomly selected mutants the sampling ratio, and the resulting set of mutants a sample.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Operator-based and random mutant selection: Better together

Zhang

Gligoric

Marinov

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation testing is a powerful methodology for evaluating the quality of a test suite. However, the methodology is also very costly, as the test suite may have to be executed for each mutant. Selective mutation testing is a well-studied technique to reduce this cost by selecting a subset of all mutants, which would otherwise have to be considered in their entirety. Two common approaches are operator-based mutant selection, which only generates mutants using a subset of mutation operators, and random mutant selection, which selects a subset of mutants generated using all mutation operators. While each of the two approaches provides some reduction in the number of mutants to execute, applying either of the two to medium-sized, realworld programs can still generate a huge number of mutants, which makes their execution too expensive. This paper presents eight random sampling strategies defined on top of operatorbased mutant selection, and empirically validates that operatorbased selection and random selection can be applied in tandem to further reduce the cost of mutation testing. The experimental results show that even sampling only 5% of mutants generated by operator-based selection can still provide precise mutation testing results, while reducing the average mutation testing time to 6.54% (i.e., on average less than 5 minutes for this study).

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Operator-based and random mutant selection: Better together

Zhang

Gligoric

Marinov

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Self Cite

View full text Add to dashboard Cite

show abstract

“…As mutation testing is widely recognized as a heavy-weight testing methodology, many researchers aim to reduce the cost of mutation testing. Selective mutation testing techniques [13,34,55] select a representative subset of all mutants and ensure that the selected set of mutants achieves almost the same results as the whole mutant set. Howden [20] proposed weak mutation, which checks whether the test could result in a different internal state in the mutant from that in the original program rather than check the final results.…”

Section: Mutation Testingmentioning

confidence: 99%

Injecting mechanical faults to localize developer faults for evolving software

2013

Self Cite

View full text Add to dashboard Cite

This paper presents a novel methodology for localizing faults in code as it evolves. Our insight is that the essence of failure-inducing edits made by the developer can be captured using mechanical program transformations (e.g., mutation changes). Based on the insight, we present the FIFL framework, which uses both the spectrum information of edits (obtained using the existing FAULTTRACER approach) as well as the potential impacts of edits (simulated by mutation changes) to achieve more accurate fault localization. We evaluate FIFL on real-world repositories of nine Java projects ranging from 5.7KLoC to 88.8KLoC. The experimental results show that FIFL is able to outperform the stateof-the-art FAULTTRACER technique for localizing failureinducing program edits significantly. For example, all 19 FIFL strategies that use both the spectrum information and simulated impact information for each edit outperform the existing FAULTTRACER approach statistically at the significance level of 0.01. In addition, FIFL with its default settings outperforms FAULTTRACER by 2.33% to 86.26% on 16 of the 26 studied version pairs, and is only inferior than FAULTTRACER on one version pair.

show abstract

A systematic review of approaches for testing concurrent programs

Arora

Bhatia

Singh

2015

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYConcurrent programs are replacing the sequential programs as they utilize the true capabilities of multicore architecture. The extensive use of multicore systems and multithreaded paradigms warrants more attention to the testing of the concurrent programs. The testing concurrent program is not a new field as it has been more than 40 years because the first problem related to the testing concurrent program was addressed by the researchers. The field covers various domains, which include concurrency problems, testing approaches, techniques, graphical representations, tools, and subject systems. This paper aims at providing an overview of research in the domain of testing concurrent programs by classifying it into eight categories: (a) reachability testing, (b) structural testing, (c) model-based testing, (d) mutation-based testing, (e) slicing-based testing, (f) formal methods, (g) random testing, and (h) search-based testing. The survey is focused on the techniques applied, methodologies followed, and tools used in these aforementioned approaches. Furthermore, the gaps are also identified in different approaches. The paper concludes with the consolidation of various testing parameters along with the future directions.

show abstract

Selective mutation testing for concurrent code

Cited by 53 publications

References 44 publications

Operator-based and random mutant selection: Better together

Operator-based and random mutant selection: Better together

Injecting mechanical faults to localize developer faults for evolving software

A systematic review of approaches for testing concurrent programs

Contact Info

Product

Resources

About