Using memetic algorithm for robustness testing of contract-based software models

Abstract

Graph Transformation System (GTS) can formally specify the behavioral aspects of complex systems through graph-based contracts. Test suite generation under normal conditions from GTS specifications is a task well-suited to evolutionary algorithms such as Genetic and Particle Swarm Optimization (PSO) metaheuristics. However, testing the vulnerabilities of a system under unexpected events such as invalid inputs is essential. Furthermore, the mentioned global search algorithms tend to make big jumps in the system's state-space that are not concentrated on particular test goals. In this paper, we extend the HGAPSO approach into a cost-aware Memetic Algorithm (MA) by making small local changes through a proposed local search operator to optimize coverage score and testing costs. Moreover, we test GTS specifications not only under normal events but also under unexpected situations. So, three coverage-based testing strategies are investigated, including normal testing, robustness testing, and a hybrid strategy. The effectiveness of the proposed test generation algorithm and the testing strategies are evaluated through a type of mutation analysis at the model-level. Our experimental results show that 1) the hybrid testing strategy outperforms normal and robustness testing strategies in terms of fault-detection capability, 2) the robustness testing is the most cost-efficient strategy, and 3) the proposed MA with the hybrid testing strategy outperforms the state-of-the-art global search algorithms.