Extended Covering Arrays for Sequence Coverage

Sheng, Yunlong; Chao, Sun; Jiang, Shouda; Wei, Chang'an

doi:10.3390/sym10050146

Cited by 11 publications

(5 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Combinatorial testing with constraints on the order in which values and combinations are applied in tests was analyzed extensively in [5] . Extended covering arrays that consider the sequence of values in each test were defined in [6] . Combination sequences were studied in [20] , combining configuration and the order of combinations while also considering constraints.…”

Section: Related Workmentioning

confidence: 99%

Ordered t-way Combinations for Testing State-based Systems

Kuhn¹

2022

View full text Add to dashboard Cite

Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if code has been exercised sufficiently to ensure confidence in its functions. Measures are needed to ensure that relevant combinations of input values have been tested with adequate diversity of ordering to ensure correct operation. Combinatorial testing and combinatorial coverage measures have been applied to many types of applications but have some deficiencies for verifying and testing state-based systems where the response depends on both input values and the current system state. In such systems, internal states change as input values are processed. Examples include network protocols, which may be in listening, partial connection, full connection, disconnected, and many other states depending on the values of packet fields and the order of packets received. Similarly, merchant account balances in credit card systems change continuously as transactions are processed. This publication introduces a notion of ordered t-way combinations, proves a result regarding the construction of adequate blocks of test inputs, and discusses the application of the results to verify and test state-based systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Ordered t-way Combinations for Testing State-based Systems

Kuhn¹

2022

View full text Add to dashboard Cite

show abstract

“…Later versions allowed more than one triggering in a test case and added support for advanced constraints and other features [47,17,8]. Current sequence testing methods consist of a two-step method: the first step is to generate a list of all relevant sequences of length t (called 'Target Sequences') and the second step is to generate test cases to cover the list of all target sequences (called 'Test Sequences').…”

Section: Ctd -Related Workmentioning

confidence: 99%

“…In the test sequence generation step, the paper starts by using a greedy algorithm that handles constraints between two events, then a transition label system is proposed to represent the SUTs' requirements and graph path methods are used in order to find the optimal valid test cases. Based on this work, additional work has been done to expand the language of constraints by, e.g., adding the possibility of contiguous values [47] and by allowing more complex relationships between more than two factors [17]. Two main problems that remain open are the ability to model the SUTs' requirements in a way that allows the creation of valid test cases based on t-way testing and that the solution will be effective at run time and size, otherwise the solution will not be applicable in complex systems.…”

Section: Ctd -Related Workmentioning

confidence: 99%

“…This method has two steps, such as presented in Kuhn's work (Target and Testing Sequences), but unlike their work our focus is representing the system model in the solver, and doing so as a regular expression (RegEx). The first and second stages in our methodology harness and utilize the capabilities of BP, and the third stage is based on Yu et al [57] in the Kuhn group works and his followers [47,17,8], while in our early work we suggested solving it using a 'z3' solver. Our suggested approach positions a number of advantages over what has been presented so far.…”

Section: Test Case Coverage -Our Suggestionmentioning

confidence: 99%

See 1 more Smart Citation

Testing Reactive Systems Using Behavioural Programming, a Model Centric Approach

Weiss¹

2021

Preprint

View full text Add to dashboard Cite

Testing is a significant aspect of software development. As systems become complex and their use becomes critical to the security and the function of society, the need for testing methodologies that ensure reliability and detect faults as early as possible becomes critical. In academia and in industry, different methods are being developed for improving the testing process. The most promising approach is the model-based approach where a model is developed that defines how the system is expected to behave and how it is meant to react. The tests are derived from the model and an analysis of the test results is conducted based on it.In the proposed doctoral research, we will investigate the prospects of using the Behavioural Programming (BP) modeling approach as an enabler for a model-based testing approach that we will develop. We will develop a natural language (textual and/or graphical) for representing the requirements. The model users will create with our language will be fed to algorithms that we will develop. This includes algorithms for the automatic creation of minimal sets of test cases that cover all of the system's requirements, algorithms for analyzing the results of the tests, and other tools that support the testing process.The focus of our methodology will be to find faults caused by the interaction between different requirements in ways that are difficult for the testers to detect. Specifically, we will focus our attention to concurrency issues such as deadlocks and logical race condition. We will use a variety of methods that are made possible by BP, such as non-deterministic execution of scenarios and use of in-code model-checking for building test scenarios and for finding minimal coverage of the test scenarios for the system requirements using Combinatorial Test Design (CTD) methodologies. We will develop a proof-of-concept tool kit which will allow us to demonstrate and evaluate the above mentioned capabilities. We will compare the performance of our tools with the performance of manual testers and of other model-based tools using comparison criteria that we will define and develop.

show abstract

“…As to find a test suite of minimum size which satisfies the first condition of PITCTSs is a NP-complete problem [25], to find a PITCTS of minimum size is still NPcomplete. To construct PITCTSs which have sizes as small as possible, Particle Swarm Optimization (PSO) [26], [27] can play an important role.…”

Section: B Parallel Input Factorsmentioning

confidence: 99%

Constructing Test Suites for Real-Time Embedded Systems Under Input Timing Constraints

2019

Self Cite

View full text Add to dashboard Cite

Testing of real-time embedded systems (RTESs) under input timing constraints is a critical issue. Models which can specify timing constraints have respective merits and demerits and test suites which can cover more input possibilities and detect more faults under input timing constraints are worthy of study. In this paper, clocked computation tree logic which is used to specify input timing constraints is presented. Neighbor covering arrays and parallel input time correlation test suites are introduced to test RTESs under serial and parallel input timing constraints. Three algorithms are described in generating test suites, respectively, and corresponding random testing-based algorithms which are used as baselines for comparison are introduced. Benchmarks with different configurations are conducted to evaluate the algorithms' performance. Three real-world RTESs are tested with the test suites described in this paper, respectively. The test results show that random testing may omit some neighbor input time point combinations as the randomness and increase test suite scales. This fact may lead to the omission of some faults and heavy costs. Therefore, the proposed test suites are more effective and efficient for testing RTESs under input timing constraints. INDEX TERMS Real-time embedded systems, input timing constraints, test suite generation, random testing.

show abstract

Extended Covering Arrays for Sequence Coverage

Cited by 11 publications

References 37 publications

Ordered t-way Combinations for Testing State-based Systems

Ordered t-way Combinations for Testing State-based Systems

Testing Reactive Systems Using Behavioural Programming, a Model Centric Approach

Constructing Test Suites for Real-Time Embedded Systems Under Input Timing Constraints

Contact Info

Product

Resources

About