A systematic review on search‐based test suite reduction: State‐of‐the‐art, taxonomy, and future directions

Abstract

Regression testing remains a promising research area for the last few decades. It is a type of testing that aims at ensuring that recent modifications have not adversely affected the software product. After the introduction of a new change in the system under test, the number of test cases significantly increases to handle the modification. Consequently, it becomes prohibitively expensive to execute all of the generated test cases within the allocated testing time and budget. To address this situation, the test suite reduction (TSR) technique is widely used that focusses on finding a representative test suite without compromising its effectiveness such as fault‐detection capability. In this work, a systematic review study is conducted that intends to provide an unbiased viewpoint about TSR based on various types of search algorithms. The study's main objective is to examine and classify the current state‐of‐the‐art approaches used in search‐based TSR contexts. To achieve this, a systematic review protocol is adopted and, the most relevant primary studies (57 out of 210) published between 2007 and 2022 are selected. Existing search‐based TSR approaches are classified into five main categories, including evolutionary‐based, swarm intelligence‐based, human‐based, physics‐based, and hybrid, grounded on the type of employed search algorithm. Moreover, the current work reports the parameter settings according to their category, the type of considered operator(s), and the probabilistic rate that significantly impacts on the quality of the obtained solution. Furthermore, this study describes the comparison baseline techniques that support the empirical comparison regarding the cost‐effectiveness of a search‐based TSR approach. Finally, it isconcluded that search‐based TSR has great potential to optimally solve the TSR problem. In this regard, several potential research directions are outlined as useful for future researchers interested in conducting research in the TSR domain.

Empirical Evaluation of Tetrad Optimization Methods for Test Case Selection and Prioritization

“…Over the three decades of research, many techniques and algorithms have been developed and tested for TCSP (11)(12)(13)(14) . Most of the studies focus on suggesting and implementing new techniques, or survey the existing similar types of approaches.…”

Maximizing Test Coverage for Security Threats Using Optimal Test Data Generation

“…However, due to the constraints of the testing project's budget, manpower, and testing time, it would be unrealistic to expect perfect and faultless software to successfully complete the project. Therefore, it is essential to manage the testing cost to reduce the overall development budget and time [2]. As a result, rather than spending a large budget pursuing a flawless software system, most software developers would make a compromised testing plan [3].…”