The time it takes software systems to be tested is usually long. Search-based test selection has been a widely investigated technique to optimize the testing process. In this paper, we propose a set of seeding strategies for the test case selection problem that generate the initial population of pareto-based multi-objective algorithms, with the goals of (1) helping to find an overall better set of solutions and (2) enhancing the convergence of the algorithms. The seeding strategies were integrated with four state-of-the-art multi-objective search algorithms and applied into two contexts where regression-testing is paramount: (1) Simulation-based testing of Cyber-Physical Systems and (2) Continuous Integration. For the first context, we evaluated our approach by using six fitness function combinations and six independent case studies, whereas in the second context we derived a total of six fitness function combinations and employed four case studies. Our evaluation suggests that some of the proposed seeding strategies are indeed helpful for solving the multi-objective test case selection problem. Specifically, the proposed seeding strategies provided a higher convergence of the algorithms towards optimal solutions in 96% of the studied scenarios and an overall cost-effectiveness with a standard search budget in 85% of the studied scenarios.

The time it takes software systems to be tested is usually long. This is often caused by the time it takes the entire test suite to be executed. To optimize this, regression test selection approaches have allowed for improvements to the cost-effectiveness of verification and validation activities in the software industry. In this area, multi-objective algorithms have played a key role in selecting the appropriate subset of test cases from the entire test suite. In this paper, we propose a set of seeding strategies for the test case selection problem that generate the initial population of multi-objective algorithms. We integrated these seeding strategies with an NSGA-II algorithm for solving the test case selection problem in the context of simulation-based testing. We evaluated the strategies with six case studies and a total of 21 fitness combinations for each case study (i.e., a total of 126 problems). Our evaluation suggests that these strategies are indeed helpful for solving the multi-objective test case selection problem. In fact, two of the proposed seeding strategies outperformed the NSGA-II algorithm without seeding population with statistical significance for 92.8 and 96% of the problems. CCS CONCEPTS• Software and its engineering → Software testing and debugging;

No abstract

As systems evolve, their embedded software needs constantly to be refactored. Moreover, given the different needs of different customers, embedded systems require to be customizable. The variability of these systems is large, and requires automated testing solutions. In this paper we propose a methodology that automatically generates validation environments for highly configurable embedded software that is being refactored. The method has allowed for systematically testing a real-world industrial case study involving the software in charge of controlling the doors of an elevator. Finally, we extract the lessons learned from its application.

Simulation models are frequently used to model, simulate and test complex systems (e.g., Cyber-Physical Systems (CPSs)). To allow full test automation, test cases and test oracles are required. Safety standards (e.g., the ISO 26262) highly recommend that the test cases of systems like CPSs are associated to requirements. As a result, typically, test cases that need to cover specific requirements are manually generated in the context of simulation models. This is, of course, a time-consuming and non-systematic process. However, the current practice lacks tools that generate test cases by considering functional requirements for simulation-based testing. In this short paper we propose a Domain-Specific Language (DSL) for specifying requirements for simulation-based testing in an easy manner. These files are later parsed by an automatic test generation algorithm, which generates test cases that follow the ASAM-XiL standard. The tool was integrated with two professional tools: (1) SYNECT from dSPACE and (2) xMOD from FEV. An initial validation was also performed with an industrial simulation model from YASA motors.