An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage

Li, Nan; Praphamontripong, Upsorn; Offutt, Jeff

doi:10.1109/icstw.2009.30

Cited by 104 publications

(94 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…That paper also introduced selective mutation, which reduces the number of mutants to be proportional to the number of variable references ( ( )). More recently, Li et al found that although selective mutation has far more test requirements than the edge-pair, all-uses and prime path criteria, it ultimately needs fewer tests [11]. This implies that many mutants are redundant, encouraging us to believe that mutation testing can be effective with fewer mutants.…”

Section: Introductionmentioning

confidence: 99%

Empirical Evaluation of the Statement Deletion Mutation Operator

Deng

Offutt

2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation analysis is widely considered to be an exceptionally effective criterion for designing tests. It is also widely considered to be expensive in terms of the number of test requirements and in the amount of execution needed to create a good test suite. This paper posits that simply deleting statements, implemented with the statement deletion (SDL) mutation operators in Mothra, is enough to get very good tests. A version of the SDL operator for Java was designed and implemented inside the muJava mutation system. The SDL operator was applied to 40 separate Java classes, tests were designed to kill the non-equivalent SDL mutants, and then run against all mutants.

show abstract

Section: Introductionmentioning

confidence: 99%

Empirical Evaluation of the Statement Deletion Mutation Operator

Deng

Offutt

2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, Li et al found that mutation had far more test requirements than the edge-pair, all-uses and prime path criteria, even though it ultimately yielded fewer tests [29]. The conclusion from this result is that the mutants somehow "overlap" in the tests needed to kill them, and probably in their ability to find faults.…”

Section: Improving Logic-based Testing With Mutationmentioning

confidence: 70%

Improving logic-based testing

Kaminski

Ammann

Offutt

2013

Journal of Systems and Software

Self Cite

View full text Add to dashboard Cite

Logic-based testers design tests from logical expressions that appear in software artifacts such as source code, design models, and requirements specifications. This paper presents three improvements to logic-based test design. First, in the context of mutation testing, we present fault hierarchies for the six relational operators. Applying the ROR mutation operator causes each relational operator to generate seven mutants per clause. The fault hierarchies show that only three of these seven mutants are needed. Second, we show how to bring the power of the ROR operator to logic-based test criteria such as the widely used Multiple Condition-Decision Coverage (MCDC) test criterion. Third, we present theoretical results supported by empirical data that show that the more recent coverage criterion of minimal-MUMCUT can find significantly more faults than MCDC. The paper has three specific recommendations: (1) Change the way the ROR mutation operator is defined in existing and future mutation systems; (2) Augment logic-based test criteria to incorporate relational operator replacement from mutation; (3) Replace the use of MCDC with minimal-MUMCUT, both in practice and in standards documents like FAA-DO178B.

show abstract

“…Despite the long time that has elapsed since many coverage criteria have been proposed, there is still no proper means of accurately measuring the effectiveness of different criteria [2], [15]. Although several researchers addressed the evaluation of effectiveness of coverage criteria and comparing them relatively, the results are not sufficient to guide practitioners in the industrial field [18].…”

Section: B Undetermined Effectiveness Of Coverage Criteriamentioning

confidence: 95%

“…The relative effectiveness of non-comparable criteria using the subsumption relation is another issue that needs to be resolved. In conclusion, additional research is needed to provide scientific evidence of the effectiveness of coverage criteria [4], [10], [11], [15].…”

Section: B Undetermined Effectiveness Of Coverage Criteriamentioning

confidence: 99%

“…To help testers decide which coverage criteria to use and when, several evaluation studies have been carried out by researchers [12]- [15] and more studies are being carried out at the moment. Despite the importance of these studies in shedding a light on the importance of analyzing coverage criteria, neither of these studies provides a final answer on which criteria to use in some certain context.…”

Section: B Test Coveragementioning

confidence: 99%

See 1 more Smart Citation

Open Problems in Software Test Coverage

Alrmuny¹

2014

LNSE

View full text Add to dashboard Cite

Abstract-Software testing is concerned with the production and maintenance of high quality systems. Test coverage is one major component in software testing. It gives a meaningful description of how much testing is required. The level of test coverage is an indicator of testing thoroughness, which in turn helps testers decide when to stop testing. Several studies addressed coverage criteria effectiveness and adequacy. However, there are several interesting open problems. We identify nine open problems in test coverage and discuss what issues need to be addressed. The objective of this paper is to direct the attention of researchers and testers toward the areas that need to be investigated.

show abstract

An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage

Cited by 104 publications

References 20 publications

Empirical Evaluation of the Statement Deletion Mutation Operator

Empirical Evaluation of the Statement Deletion Mutation Operator

Improving logic-based testing

Open Problems in Software Test Coverage

Contact Info

Product

Resources

About