State-Based Testing: Industrial Evaluation of the Cost-Effectiveness of Round-Trip Path and Sneak-Path Strategies

Holt, Nina Elisabeth; Torkar, Richard; Briand, Lionel C.; Hansen, Kai T.

doi:10.1109/issre.2012.17

Cited by 10 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Discussions: Our results support the recommendation of Binder [3] and the conclusions drawn by Mouchawrab et al [17] and Holt et al [14]: Testing sneak paths (in our case as boundaries of product line variants) is an essential component of state-based testing and drastically increases fault detection capabilities.…”

Section: Resultssupporting

confidence: 90%

“…Furthermore the results indicate that sneak path testing is a necessary step in state-based testing due to the same observations made by Holt et al [14]: 1) The proportion of sneak paths in the collected fault data was high (61,5 %), and 2) the presence of sneak paths is undetectable by conformance testing.…”

Section: Resultssupporting

confidence: 79%

“…Sneak path testing aims at verifying the absence of sneak paths and at showing that the software under test handles them in a correct way. Several studies showed that sneak path testing improves the fault detection capabilities [4,14,17]. However, the eort spent for sneak path testing is considerably high.…”

Section: Resultsmentioning

confidence: 99%

“…More formally speaking, we dene a product's boundary by boundary transitions over UML State Machines. We dene a boundary transition bt, where S be the set of states and T be the set of transitions in a PL model and t(s, s ) be a transition from state s to s as: is costly [14]. Here, we propose a method to reduce test eort by stimulating the PUT with unexpected events only if its active state has at least one boundary transition.…”

Section: Boundary Transitionsmentioning

confidence: 99%

See 3 more Smart Citations

Automated Test Design for Boundaries of Product Line Variants

Weiβleder

Wartenberg

Lackner

2015

Testing Software and Systems

View full text Add to dashboard Cite

Abstract. Developing product lines is usually more ecient than developing single products because of the reuse of single components. Testing, however, has to consider complete, integrated systems. To prevent testing every product on system level, the whole product line should be analyzed with the aim of selecting distinguishing product behavior and a minimum of system products to test. In this paper, we present a model-based test design approach for testing the selected behavior of products, but also their deselected behavior. A major challenge of this approach is that the deselected behavior of a product is often not part of its behavioral model. Thus, we use the variability model to transform the behavioral model so that showing the exclusion of the deselected behavior is also covered by tests. We present the approach, a corresponding prototypical implementation, and our experiences using a set of examples.

show abstract

Section: Resultssupporting

confidence: 90%

Section: Resultssupporting

confidence: 79%

Section: Resultsmentioning

confidence: 99%

Section: Boundary Transitionsmentioning

confidence: 99%

See 2 more Smart Citations

Automated Test Design for Boundaries of Product Line Variants

Weiβleder

Wartenberg

Lackner

2015

Testing Software and Systems

View full text Add to dashboard Cite

show abstract

“…Many testing strategies can be applied to state-based testing. A state-based testing strategy that proved to provide a balance between effectiveness at finding faults and cost is the RTP criterion [9]. This criterion is satisfied when all RTPs (i.e., loops) are exercised [5].…”

Section: Test Strategiesmentioning

confidence: 99%

Comparing transition trees test suites effectiveness for different mutation operators

Khalil

Labiche

2020

Proceedings of the 11th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

View full text Add to dashboard Cite

Research demonstrated that faults seeded mutation using operators can be representative of faults in real systems. In this paper, we study the relationship between the different operators used to insert mutants in the fault domain of the system under test and the effectiveness of different state machine test suites at killing those mutants. We are particularly interested in the effectiveness of two interrelated state machine testing strategies at finding different types of faults. Those are the round-trip paths strategy and the transition tree strategy. Using empirical evaluation, we compare the effectiveness of more than two thousand unique test suites at killing mutants seeded using eight different mutation operators. We perform experiments on four experimental objects and provide qualitative analysis of the results. We conclude that neither of the two studied strategies is more effective than the other at killing a certain type of mutants. However, the structure of the finite state machine and the nature of the system under test affect the type of faults detected by the different testing strategies.

show abstract