Multi-objective black-box test case selection for system testing

Lachmann, Remo; Felderer, Michael; Nieke, Manuel; Schulze, Sandro; Seidl, Christoph; Schaefer, Ina

doi:10.1145/3071178.3071189

Cited by 19 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper sets on various existing methods for de termining a model of an SUT (see Section 11), reducing the potential test space (Section 11-A), deriving attacks and generate test cases based on attack trees (or similar -see Section III). The related work for each part of the proposed system is cited in the respective sections; high level overviews of model-based and black-box security testing can be found at [4] and [5], respectively.…”

Section: A Related Work and Contributionmentioning

confidence: 99%

A Model-Driven Methodology for Automotive Cybersecurity Test Case Generation

Marksteiner

Priller

2021

2021 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)

View full text Add to dashboard Cite

Through international regulations (most promi nently the latest UNECE regulation) and standards, the already widely perceived higher need for cybersecurity in automotive systems has been recognized and will mandate higher efforts for cybersecurity engineering. The UNECE also demands the effectiveness of these engineering to be verified and validated through testing. This requires both a significantly higher rate and more comprehensiveness of cybersecurity testing that is not effectively to cope with using current, predominantly manual, automotive cybersecurity testing techniques. To allow for comprehensive and efficient testing at all stages of the automotive life cycle, including supply chain parts not at band, and to facilitate efficient third party testing, as well as to test under real-world conditions, also methodologies for testing the cybersecurity of vehicular systems as a black box are necessary. This paper therefore presents a model and attack tree-based approach to (semi-)automate automotive cybersecurity testing, as well as considerations for automatically black box-deriving models for the use in attack modeling.

show abstract

Section: A Related Work and Contributionmentioning

confidence: 99%

A Model-Driven Methodology for Automotive Cybersecurity Test Case Generation

Marksteiner

Priller

2021

2021 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)

View full text Add to dashboard Cite

show abstract

“…In the last few years, several new approaches have been proposed to adapt the test case selection problem to different emergent areas, including defence software [33,57], or compute-intensive CPSs [6,7]. Most multi-objective test selection approaches aim at proposing effective objective functions and study whether they act as a reasonable surrogate for fault detection capabilities [6,7,36,38,68]. Unlike all these studies, our approach aims at comparing how different strategies for seeding the initial population perform in the context of multi-objective test case selection algorithms.…”

Section: Related Workmentioning

confidence: 99%

“…In the last few years, test selection based on evolutionary algorithms have gained important attention. Most of them have focused either on comparing (1) which adequacy criteria could fit best for integrating it in the fitness functions [6,7,36,38,68] or (2) which algorithm performs best when selecting test cases (when having one specific fitness function) [10,62,66,67]. Additionally, most of them compare their approaches with a baseline algorithm, such as, Random Search (RS) [6,7,10,62,66,67] or Greedy [68].…”

mentioning

confidence: 99%

Seeding strategies for multi-objective test case selection

Arrieta

Agirre

Sagardui

2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

The time it takes software systems to be tested is usually long. This is often caused by the time it takes the entire test suite to be executed. To optimize this, regression test selection approaches have allowed for improvements to the cost-effectiveness of verification and validation activities in the software industry. In this area, multi-objective algorithms have played a key role in selecting the appropriate subset of test cases from the entire test suite. In this paper, we propose a set of seeding strategies for the test case selection problem that generate the initial population of multi-objective algorithms. We integrated these seeding strategies with an NSGA-II algorithm for solving the test case selection problem in the context of simulation-based testing. We evaluated the strategies with six case studies and a total of 21 fitness combinations for each case study (i.e., a total of 126 problems). Our evaluation suggests that these strategies are indeed helpful for solving the multi-objective test case selection problem. In fact, two of the proposed seeding strategies outperformed the NSGA-II algorithm without seeding population with statistical significance for 92.8 and 96% of the problems. CCS CONCEPTS• Software and its engineering → Software testing and debugging;

show abstract

“…Their results outperform a random approach. In previous work, we defined a multi-objective test case selection approach for black-box testing [18]. We define seven different objectives to be optimized using genetic algorithms.…”

Section: Related Workmentioning

confidence: 99%

12.4 - Machine Learning-Driven Test Case Prioritization Approaches for Black-Box Software Testing

Lachmann¹

2018

Proceeding - Ettc2018

View full text Add to dashboard Cite

Multi-objective black-box test case selection for system testing

Cited by 19 publications

References 31 publications

A Model-Driven Methodology for Automotive Cybersecurity Test Case Generation

A Model-Driven Methodology for Automotive Cybersecurity Test Case Generation

Seeding strategies for multi-objective test case selection

12.4 - Machine Learning-Driven Test Case Prioritization Approaches for Black-Box Software Testing

Contact Info

Product

Resources

About