Coverage-Directed Test Generation with Model Checkers: Challenges and Opportunities

Devaraj, George; Heimdahl, Mats P. E.; Liang, Donglin

doi:10.1109/compsac.2005.66

Cited by 11 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7 We do not reject H01 for the DWM 1 case example when using MC/DC coverage and the output-only oracle, nor do we reject H01 for the Docking Approach case example. 8 In our previous work we found the opposite effect [34].…”

Section: Evaluation Of Rq1 and Rq2mentioning

confidence: 92%

See 1 more Smart Citation

On the Danger of Coverage Directed Test Case Generation

Staats

Gay

Whalen

et al. 2012

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

Abstract. In the avionics domain, the use of structural coverage criteria is legally required in determining test suite adequacy. With the success of automated test generation tools, it is tempting to use these criteria as the basis for test generation. To more firmly establish the effectiveness of such approaches, we have generated and evaluated test suites to satisfy two coverage criteria using counterexample-based test generation and a random generation approach, contrasted against purely random test suites of equal size.Our results yield two key conclusions. First, coverage criteria satisfaction alone is a poor indication of test suite effectiveness. Second, the use of structural coverage as a supplement-not a target-for test generation can have a positive impact. These observations points to the dangers inherent in the increase in test automation in critical systems and the need for more research in how coverage criteria, generation approach, and system structure jointly influence test effectiveness.

show abstract

Section: Evaluation Of Rq1 and Rq2mentioning

confidence: 92%

“…Other measurements exist, e.g., the time required to generate and/or execute tests [34]. We chose size to be favorable towards directed test generation.…”

Section: Threats To Validitymentioning

confidence: 99%

On the Danger of Coverage Directed Test Case Generation

Staats

Gay

Whalen

et al. 2012

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

show abstract

“…If a property is not satisfied, the model checker attempts to generate a counterexample in the form of a trace as a sequence of states [5]. The idea of using traces of counterexamples as test sequences is not new and investigated in various works [5], [6], [7], [8], [9]. Other approaches, such as mutant-based model checking to ensure safety properties [10], exists in the literature.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Towards Test Case Generation for Synthesizable VHDL Programs Using Model Checker

Ayav

Tuğlular

Belli

2010

2010 Fourth International Conference on Secure Software Integration and Reliability Improvement Companion

View full text Add to dashboard Cite

Abstract-VHDL programs are often tested by means of simulations, relying on test benches written intuitively. In this paper, we propose a formal approach to construct test benches from system specification. To consider the real-time properties of VHDL programs, we first transform them to timed automata and then perform model checking against the properties designated from the specification. Counterexamples returned from the model checker serve as a basis of test cases, i.e., they are used to form a test bench. The approach is demonstrated and complemented by a simple case study.

show abstract

“…In a consequent experiment, Devaraj et al [108] showed that while coverage criteria are suitable for analysis purposes, there are problems when using them for test case generation. The identified problem is that coverage of a trap property does not guarantee that a considered part of the specification is actually executed by the resulting test case.…”

Section: Quality Concerns For Coverage Based Testingmentioning

confidence: 99%

Testing with model checkers: a survey

Fraser

Wotawa

Ammann

2008

Software Testing Verif & Rel

144

View full text Add to dashboard Cite

Abstract.About a decade after the initial proposal to use model checkers for the generation of test cases we take a look at the achievements in this field of research. Model checkers are formal verification tools, capable of providing counterexamples to violated properties. Normally, these counterexamples are meant to guide an analyst when searching for the root cause of a property violation. They are, however, also very useful as test cases. Many different approaches have been presented, many problems have been solved, yet many issues remain. This survey paper reviews the state of the art in testing with model checkers.

show abstract

Coverage-Directed Test Generation with Model Checkers: Challenges and Opportunities

Cited by 11 publications

References 10 publications

On the Danger of Coverage Directed Test Case Generation

On the Danger of Coverage Directed Test Case Generation

Towards Test Case Generation for Synthesizable VHDL Programs Using Model Checker

Testing with model checkers: a survey

Contact Info

Product

Resources

About