Reconciling perspectives of software logic testing

Abstract

Many software logic test coverage criteria have emerged over the past several years; however, they are scattered throughout the literature. The goal of this paper is to describe the various logic tests and explain their rationale in a centralized location in order to aid software testers in their decisions about which to implement. Each logic test is examined in terms of its minimum and maximum test sizes, subsumption relationship to other logic tests, and its fault detection capability. It is shown that although semantic tests generally have a smaller test size, syntactic tests are better in terms of fault detection capability. Furthermore, it is shown that the syntactic tests subsume their semantic counterparts. Two new software faults are introduced to Kuhn's fault hierarchy, namely the term insertion fault and the term negation fault, and the fault detection capability of the logic tests with respect to these faults is examined. A new software testing criterion called multiple unique true point/near false point (MUTP/NFP) is also presented, with a description of its benefits and limitations. Specifically, a detailed comparison of the MUTP/NFP criterion and the CUTPNFP/MNFP/MUTP criterion is given. Copyright © 2008 John Wiley & Sons, Ltd.

Generating Minimal Fault Detecting Test Suites for Boolean Expressions

“…This means that less test cases are required for the original predicate (as proved also by Kaminski et al [24]). Consider for example the expression ϕ = a ∨ (b ∧ c) containing 3 variables and 3 literals.…”

Improving logic-based testing

“…As said in section 2, RACC tests are only guaranteed to detect faults of type ENF and TNF [24]. However, in practice, RACC tests do indeed detect some other faults, even though they are not guaranteed to detect such faults.…”

“…MCDC is considered to be a semantic coverage criterion [24], whereas mutation is considered to be syntactic. Semantic criteria generate test requirements independently of how the predicates are written, whereas syntactic criteria do not.…”

“…The third improvement is a demonstration, both theoretical and empirical, of how minimal-MUMCUT [24] is significantly stronger than MCDC. Minimal-MUMCUT is proved to detect more faults than MCDC, with little additional cost.…”

“…They are fully defined in Lau and Yu's paper [28]. Despite the widespread use and confidence in MCDC, MCDC tests are only guaranteed to detect TNF and ENF faults [24]! This fault hierarchy has been used to develop alternative logic-based test criteria, the most powerful being MUMCUT [4].…”