Applying Combinatorial Testing to the Siemens Suite

Ghandehari, Laleh Shikh Gholamhossein; Bourazjany, Mehra N.; Lei, Yu; Kacker, Raghu N.; Kuhn, D. Richard

doi:10.1109/icstw.2013.47

Cited by 25 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the one hand, the effectiveness of t-way testing has been shown by a number of empirical studies [4], [17], [20], [27], [30], [42]. On the other hand, coverage-based software testing could raise an open question whether the coverage is actually useful for real fault detection [21].…”

Section: Discussionmentioning

confidence: 99%

Distance-Integrated Combinatorial Testing

Choi

Artho

Kitamura

et al. 2016

2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

Abstract-This paper proposes a novel approach to combinatorial test generation, which achieves an increase of not only the number of new combinations but also the distance between test cases. We applied our distance-integrated approach to a stateof-the-art greedy algorithm for traditional combinatorial test generation by using two distance metrics, Hamming distance, and a modified chi-square distance. Experimental results using numerous benchmark models show that combinatorial test suites generated by our approach using both distance metrics can improve interaction coverage for higher interaction strengths with low computational overhead.

show abstract

Section: Discussionmentioning

confidence: 99%

Distance-Integrated Combinatorial Testing

Choi

Artho

Kitamura

et al. 2016

2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

show abstract

“…Hence, we first need to generate a set of abstract test cases. Some of the generated abstract tests need to be combined and enriched with glue information to form actual test cases [24]. Figure 3 shows how many parameters each modeled program contains.…”

Section: A Input Space Modelingmentioning

confidence: 99%

Combinatorial Modeling and Test Case Generation for Industrial Control Software Using ACTS

Ericsson¹,

Enoiu²

2018

2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)

View full text Add to dashboard Cite

Combinatorial testing has been suggested as an effective method of creating test cases at a lower cost. However, industrially applicable tools for modeling and combinatorial test generation are still scarce. As a direct effect, combinatorial testing has only seen a limited uptake in industry that calls into question its practical usefulness. This lack of evidence is especially troublesome if we consider the use of combinatorial test generation for industrial safety-critical control software, such as are found in trains, airplanes, and power plants.To study the industrial application of combinatorial testing, we evaluated ACTS, a popular tool for combinatorial modeling and test generation, in terms of applicability and test efficiency on industrial-sized IEC 61131-3 industrial control software running on Programmable Logic Controllers (PLC). We assessed ACTS in terms of its direct applicability in combinatorial modeling of IEC 61131-3 industrial software and the efficiency of ACTS in terms of generation time and test suite size. We used 17 industrial control programs provided by Bombardier Transportation Sweden AB and used in a train control management system. Our results show that not all combinations of algorithms and interaction strengths could generate a test suite within a realistic cut-off time. The results of the modeling process and the efficiency evaluation of ACTS are useful for practitioners considering to use combinatorial testing for industrial control software as well as for researchers trying to improve the use of such combinatorial testing techniques.

show abstract

“…In order to apply combinatorial testing, we first modeled the input parameters for the replace program. The details of the model are discussed in [ 3]. Then, ACTS [ 11] is used to generate a 2-way test set consisting of 190 tests.…”

Section: Use Casementioning

confidence: 99%

“…The first phase produces a ranking of combinations in terms of their likelihood to be failureinducing. A combination is failure-inducing, or simply inducing, if all tests containing this combination fail [ 3,5,7,12,13]. In the second phase, BEN takes a top ranked inducing combination from which a failed test and a small number of passed tests are generated.…”

Section: Introductionmentioning

confidence: 99%

BEN: A combinatorial testing-based fault localization tool

Ghandehari

Chandrasekaran

Lei

et al. 2015

2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

Self Cite

View full text Add to dashboard Cite

We present a combinatorial testing-based fault localization tool called BEN. BEN takes as input three types of information, including the subject program, the source code, an input parameter model, and a combinatorial test set created based on the input parameter model. It is assumed that the combinatorial test set has already been executed, and thus the execution status of each test is known. The output of BEN is a ranking of statements in terms of their likelihood to be faulty. In the fault localization process, a small number of additional tests are generated by BEN and need to be executed by the user. In this paper, we present the major user scenarios and the highlevel design of BEN. BEN is implemented in Java and provides a graphical user interface that provides friendly access to the tool.

show abstract

Applying Combinatorial Testing to the Siemens Suite

Cited by 25 publications

References 10 publications

Distance-Integrated Combinatorial Testing

Distance-Integrated Combinatorial Testing

Combinatorial Modeling and Test Case Generation for Industrial Control Software Using ACTS

BEN: A combinatorial testing-based fault localization tool

Contact Info

Product

Resources

About