Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach

Cohen, Myra B.; Dwyer, Matthew B.; Shi, Jiangfan

doi:10.1109/tse.2008.50

Cited by 277 publications

(264 citation statements)

References 33 publications

Supporting

Mentioning

261

Contrasting

Unclassified

Order By: Relevance

“…There is no single recommended approach available today for testing interactions between features in product lines efficiently [11], but there are many suggestions. Some of the more promising suggestions are combinatorial interaction testing [1], discussed below; a technique called ScenTED, where the idea is to express the commonalities and differences on the UML model of the product line and then derive concrete test cases by analyzing it [12]; and incremental testing, where the idea is to automatically adapt a test case from one product to the next using the specification of similarities and differences between the products [13]. Kim et al 2011 [14] presented a technique where they can identify irrelevant features for a test case using static analysis.…”

Section: Reusable Component Testingmentioning

confidence: 99%

“…Combinatorial Interaction Testing: Combinatorial interaction testing [1] is one of the most promising approaches. The benefits of this approach is that it deals directly with the feature model to derive a small set of products (a covering array) which products can then be tested using single system testing techniques, of which there are many good ones [15].…”

Section: Reusable Component Testingmentioning

confidence: 99%

“…One approach for testing product lines is combinatorial interaction testing [1]. Combinatorial interaction testing is to first construct a small set of products, called a covering array, in which interaction faults are most likely to show up and then to test these products normally.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Technique for Agile and Automatic Interaction Testing for Product Lines

Johansen¹,

Haugen²,

Fleurey³

et al. 2012

Testing Software and Systems

View full text Add to dashboard Cite

Abstract. Product line developers must ensure that existing and new features work in all products. Adding to or changing a product line might break some of its features. In this paper, we present a technique for automatic and agile interaction testing for product lines. The technique enables developers to know if features work together with other features in a product line, and it blends well into a process of continuous integration. The technique is evaluated with two industrial applications, testing a product line of safety devices and the Eclipse IDEs. The first case shows how existing test suites are applied to the products of a 2-wise covering array to identify two interaction faults. The second case shows how over 400,000 test executions are performed on the products of a 2-wise covering array using over 40,000 existing automatic tests to identify potential interactions faults.

show abstract

Section: Reusable Component Testingmentioning

confidence: 99%

Section: Reusable Component Testingmentioning

confidence: 99%

See 1 more Smart Citation

A Technique for Agile and Automatic Interaction Testing for Product Lines

Johansen¹,

Haugen²,

Fleurey³

et al. 2012

Testing Software and Systems

View full text Add to dashboard Cite

show abstract

“…It is important to notice that there exist very few studies that have tackled the automatic generation for CIT in the presence of constraints between variables [6]. In order to include properties that forbid combinations of values, CIT generation techniques have to allow the introduction of constraints in the algorithms that generate covering arrays.…”

Section: Combinatorial Interaction Testingmentioning

confidence: 99%

“…In the work of Cohen et al [6], this technique is extended to software product lines with highly configurable systems. Modeling variability in SPLs using feature models is the work of Jaring and Boschet [9] where they show that the robustness of a SPL architecture is related to the type of variability.…”

Section: Related Workmentioning

confidence: 99%

Pairwise testing of dynamic composite services

Kattepur

Sen

Baudry

et al. 2011

Proceedings of the 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems

View full text Add to dashboard Cite

Online services encapsulate enterprises, people, software systems and often operate in poorly understood environments. Using such services in tandem to predictably orchestrate a complex task is one of the principal challenges of serviceoriented computing. A composite service orchestration soliciting multiple atomic services is plagued by a number of sources of variation. For instance, availability of an atomic service and its response time are two important sources of variation. Moreover, the number of possible variations in a composite service increases exponentially with increase in the number of atomic services. Testing such a composite service presents a crucial challenge as its often very expensive to exhaustively examine the variation space. Can we effectively test the dynamic behavior of a composite service using only a subset of these variations? This is the question that intrigues us. In this paper, we first model composite service variability as a feature diagram (FD) that captures all valid configurations of its orchestration. Second, we apply pairwise testing to sample the set of all possible configurations to obtain a concise subset. Finally, we test the composite service for selected pairwise configurations for a variety of QoS metrics such as response time, data quality, and availability. Using two case studies, Car crash crisis management and eHealth management, we demonstrate that pairwise generation effectively samples the full range of QoS variations in a dynamic orchestration. The pairwise sampling technique eliminates over 99% redundancy in configurations, while still calling all atomic services at least once. We rigorously evaluate pairwise testing for the criteria such as: a) ability to sample the extreme QoS metrics of the service b) stable behavior of the extracted configurations c) compact set of configurations that can help evaluate QoS tradeoffs and d) comparison with random sampling.

show abstract

Testing Model Transformations: A Case for Test Generation from Input Domain Models

Baudry

2013

Model‐Driven Engineering for Distributed Real‐Time Systems

View full text Add to dashboard Cite

Model transformations can automate critical tasks in model-driven development. Thorough validation techniques are required to ensure their correctness. In this lecture we focus on testing model transformations. In particular, we present an approach for systematic selection of input test data. This approach is based on a key characteristic of model transformations: their input domain is formally captured in a metamodel. A major challenge for test generation is that metamodels usually model an infinite set of possible input models for the transformation. We start with a general motivation of the need for specific test selection techniques in the presence of very large and possibly infinite input domains. We also present two existing black-box strategies to systematically select test data: category-partition and combinatorial interaction testing. Then, we detail specific criteria based on metamodel coverage to select data for model transformation testing. We introduce object and model fragments to capture specific structural constraints that should be satisfied by input test data. These fragments are the basis for the definition of coverage criteria and for automatic generation of test data. They also serve to drive the automatic generation of models for testing.

show abstract

Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach

Cited by 277 publications

References 33 publications

A Technique for Agile and Automatic Interaction Testing for Product Lines

A Technique for Agile and Automatic Interaction Testing for Product Lines

Pairwise testing of dynamic composite services

Testing Model Transformations: A Case for Test Generation from Input Domain Models

Contact Info

Product

Resources

About