Comparing Multi-Point Stride Coverage and dataflow coverage

Hassan, Mohammad Mahdi; Andrews, James H.

doi:10.1109/icse.2013.6606563

Cited by 17 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, mutation testing is both difficult to apply and computationally expensive, which has led to the search for "next-best" criteria for predicting suite quality by researchers [16,19]. This effort is highly relevant to real software developers, who almost never apply mutation testing due to its complexity, expense, and the lack of tool support in many languages.…”

Section: Introductionmentioning

confidence: 99%

“…Many recent papers comparing criteria either introduce a novel criteron [19] or implement a previously proposed but never-implemented criteria [16]. While most even moderately popular languages (e.g., Java, C, C++, Python, Haskell) have (multiple) tools for measuring statement and branch coverage, relatively few non-research tools offer any variant of path coverage, much less data flow-based [20,19] or more esoteric measures such as predicate-complete test coverage measures [5,6]. This limitation is particularly important for open source development, where expensive commercial coverage tools are unlikely to be applied, and developers working together may lack a common development environment.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Code coverage for suite evaluation by developers

Gopinath

Jensen

Groce

2014

Proceedings of the 36th International Conference on Software Engineering

136

View full text Add to dashboard Cite

One of the key challenges of developers testing code is determining a test suite's quality -its ability to find faults. The most common approach is to use code coverage as a measure for test suite quality, and diminishing returns in coverage or high absolute coverage as a stopping rule. In testing research, suite quality is often evaluated by a suite's ability to kill mutants (artificially seeded potential faults). Determining which criteria best predict mutation kills is critical to practical estimation of test suite quality. Previous work has only used small sets of programs, and usually compares multiple suites for a single program. Practitioners, however, seldom compare suites -they evaluate one suite. Using suites (both manual and automatically generated) from a large set of real-world open-source projects shows that evaluation results differ from those for suite-comparison: statement (not block, branch, or path) coverage predicts mutation kills best.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Code coverage for suite evaluation by developers

Gopinath

Jensen

Groce

2014

Proceedings of the 36th International Conference on Software Engineering

136

View full text Add to dashboard Cite

show abstract

“…Test input design and generation are important aspects of software testing, directly impacting on the testing cost and effectiveness, and are often guided by things such as program structure, source code, the software specification, input/output structure, and information about previously executed tests [4], [20]. Depending on the information used, test input design and generation techniques have been broadly classified as specification based, structure based, and RT based [1], [21].…”

Section: A Oo Software Testingmentioning

confidence: 99%

A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing

et al. 2017

View full text Add to dashboard Cite

Abstract-Random testing (RT) has been identified as one of the most popular testing techniques, due to its simplicity and ease of automation. Adaptive random testing (ART) has been proposed as an enhancement to RT, improving its fault-detection effectiveness by evenly spreading random test inputs across the input domain. To achieve the even spreading, ART makes use of distance measurements between consecutive inputs. However, due to the nature of object-oriented software (OOS), its distance measurement can be particularly challenging: Each input may involve multiple classes, and interaction of objects through method invocations. Two previous studies have reported on how to test OOS at a single-class level using ART. In this study, we propose a new similarity metric to enable multiclass level testing using ART. When generating test inputs (for multiple classes, a series of objects, and a sequence of method invocations), we use the similarity metric to calculate the distance between two series of objects, and between two sequences of method invocations. We integrate this metric with ART and apply it to a set of open-source OO programs, with the empirical results showing that our approach outperforms other RT and ART approaches in OOS testing.

show abstract

“…When applying a data-flow adequacy criterion, it generally requires executing test cases against the program to determine the achieved coverage (Frankl and Weyuker, 1988;Hassan and Andrews 2013). Unfortunately, the overhead of runtime monitoring with respect to data-flow adequacy criteria is very high due to the need for profiling data accesses in the course of program executions.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, to profile the data accesses from the executions of C/C++ programs via the popular Pin framework, the overhead can be as high as almost a 100-fold slowdown [Luk et al 2005]. This high runtime overhead problem is further amplified by the fact that dataflow test adequacy criteria are demanding to be satisfied [Weyuker 1990;Hassan and Andrews 2013]. Hence, blindly applying more test cases to a program may not effectively exercise more program elements that are identified to be not covered by previous executions.…”

Section: Related Workmentioning

confidence: 99%

Improving the Effectiveness of Testing Pervasive Software via Context Diversity

Huai

Chan

Tse

2014

ACM Trans. Auton. Adapt. Syst.

View full text Add to dashboard Cite

Context-aware pervasive software is responsive to various contexts and their changes. A faulty implementation of the context-aware features may lead to unpredictable behavior with adverse effects. In software testing, one of the most important research issues is to determine the sufficiency of a test suite to verify the software under test. Existing adequacy criteria for testing traditional software, however, have not explored the dimension of serial test inputs and have not considered context changes when constructing test suites. In this article, we define the concept of context diversity to capture the extent of context changes in serial inputs and propose three strategies to study how context diversity may improve the effectiveness of the data-flow testing criteria. Our case study shows that the strategy that uses test cases with higher context diversity can significantly improve the effectiveness of existing data-flow testing criteria for context-aware pervasive software. In addition, test suites with higher context diversity are found to execute significantly longer paths, which may provide a clue that reveals why context diversity can contribute to the improvement of effectiveness of test suites.

show abstract

Comparing Multi-Point Stride Coverage and dataflow coverage

Cited by 17 publications

References 27 publications

Code coverage for suite evaluation by developers

Code coverage for suite evaluation by developers

A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing

Improving the Effectiveness of Testing Pervasive Software via Context Diversity

Contact Info

Product

Resources

About