Pareto efficient multi-objective black-box test case selection for simulation-based testing

Arrieta, Aitor; Wang, Shuai; Markiegi, Urtzi; Arruabarrena, Ainhoa; Etxeberria, Leire; Sagardui, Goiuria

doi:10.1016/j.infsof.2019.06.009

Cited by 35 publications

(127 citation statements)

References 33 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…To generate signals, we use the approach of Matlab [16,65] that encodes signals using some parameters. Specifically, each signal u u in u is captured by ⟨int u , R u , n u ⟩, where int u is the interpolation function, R u ⊆ R is the input domain, and n u is the number of control points.…”

Section: Test Generationmentioning

confidence: 99%

Mining assumptions for software components using machine learning

Gaaloul

Menghi

Nejati

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Software verification approaches aim to check a software component under analysis for all possible environments. In reality, however, components are expected to operate within a larger system and are required to satisfy their requirements only when their inputs are constrained by environment assumptions. In this paper, we propose EPIcuRus, an approach to automatically synthesize environment assumptions for a component under analysis (i.e., conditions on the component inputs under which the component is guaranteed to satisfy its requirements). EPIcuRus combines search-based testing, machine learning and model checking. The core of EPIcuRus is a decision tree algorithm that infers environment assumptions from a set of test results including test cases and their verdicts. The test cases are generated using search-based testing, and the assumptions inferred by decision trees are validated through model checking. In order to improve the efficiency and effectiveness of the assumption generation process, we propose a novel test case generation technique, namely Important Features Boundary Test (IFBT), that guides the test generation based on the feedback produced by machine learning. We evaluated EPIcuRus by assessing its effectiveness in computing assumptions on a set of study subjects that include 18 requirements of four industrial models. We show that, for each of the 18 requirements, EPIcuRus was able to compute an assumption to ensure the satisfaction of that requirement, and further, ≈ 78% of these assumptions were computed in one hour.

show abstract

Section: Test Generationmentioning

confidence: 99%

Mining assumptions for software components using machine learning

Gaaloul

Menghi

Nejati

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

show abstract

“…Several researchers reported test case selection (TCS) or test case minimization (TCM) approaches, such as literature [33][34][35]. Banias took low memory consumption into consideration that is suited for medium to large projects and proposed a dynamic programming approach for TCS issues [33].…”

Section: Software Testingmentioning

confidence: 99%

“…Banias took low memory consumption into consideration that is suited for medium to large projects and proposed a dynamic programming approach for TCS issues [33]. Arrieta et al proposed a cost-effective approach, which defined six effectiveness measures and one cost measure relying on black-box data [34]. Zhang et al applied the Uncertainty Theory and multi-objective search to test case generation (TCG) and TCM [35].…”

Section: Software Testingmentioning

confidence: 99%

Can Defect Prediction Be Useful for Coarse-Level Tasks of Software Testing?

et al. 2020

View full text Add to dashboard Cite

It is popular to use software defect prediction (SDP) techniques to predict bugs in software in the past 20 years. Before conducting software testing (ST), the result of SDP assists on resource allocation for ST. However, DP usually works on fine-level tasks (or white-box testing) instead of coarse-level tasks (or black-box testing). Before ST or without historical execution information, it is difficult to get resource allocated properly. Therefore, a SDP-based approach, named DPAHM, is proposed to assist on arranging resource for coarse-level tasks. The method combines analytic hierarchy process (AHP) and variant incidence matrix. Besides, we apply the proposed DPAHM into a proprietary software, named MC. Besides, we conduct an up-to-down structure, including three layers for MC. Additionally, the performance measure of each layer is calculated based on the SDP result. Therefore, the resource allocation strategy for coarse-level tasks is gained according to the prediction result. The experiment indicates our proposed method is effective for resource allocation of coarse-level tasks before executing ST.

show abstract

“…In the case of cyber-physical systems, these models are sometimes shipped along with the actual device, which means that analysts can now access high-fidelity simulations of their systems. Hence, much of the work on cyber-physical testing focuses on taking full advantage of high-fidelity simulators, prior to live testing [3]. For example, analysts can use the simulators for test suite minimization; i.e.…”

Section: Introductionmentioning

confidence: 99%

Faster Multi-Goal Simulation-Based Testing Using DoLesS (Domination with Least Square Approximation)

Liu¹

2021

Preprint

View full text Add to dashboard Cite

For cyber-physical systems, finding a set of test cases with the least cost by exploring multiple goals is a complex task. For example, Arrieta et al. reported that state-of-the-art optimizers struggle to find minimal test suites for this task. To better manage this task, we propose DoLesS (Domination with Least Squares Approximation) which uses a domination predicate to sort the space of possible goals to a small number of representative examples. Multi-objective domination then divides these examples into a "best" set and the remaining "rest" set. After that, DoLesS applies an inverted least squares approximation approach to learn a minimal set of tests that can distinguish best from rest in the reduced example space.DoLesS has been tested on four cyber-physical models: a tank flow model; a model of electric car windows; a safety feature of an AC engine; and a continuous PID controller combined with a discrete state machine. Comparing to the recent state-of-the-art paper attempted the same task, DoLesS performs as well or even better as state-of-the-art, while running 80-360 times faster on average (seconds instead of hours). Hence, we recommend DoLesS as a fast method to find minimal test suites for multi-goal cyberphysical systems. For replication purposes, all our code is on-line: https://github.com/hellonull123/Test_Selection_2021.

show abstract

Pareto efficient multi-objective black-box test case selection for simulation-based testing

Cited by 35 publications

References 33 publications

Mining assumptions for software components using machine learning

Mining assumptions for software components using machine learning

Can Defect Prediction Be Useful for Coarse-Level Tasks of Software Testing?

Faster Multi-Goal Simulation-Based Testing Using DoLesS (Domination with Least Square Approximation)

Contact Info

Product

Resources

About