Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.This survey aims to collect and consider papers that deal with three regression testing techniques: test suite minimization, RTS and test case prioritization. Our intention is not to undertake a systematic review, but rather to provide a broad state-of-the-art view on these related fields. Many different approaches have been proposed to aid regression testing, which has resulted in a body of literature that is spread over a wide variety of domains and publication venues. The majority of surveyed literature has been published in the software engineering domain, and especially in the software testing and software maintenance literature. However, the regression testing literature also overlaps with those of programming language analysis, empirical software engineering and software metrics.Therefore, the paper selection criteria on which this survey is based are the problems considered in papers, while focusing on the specific topics of minimization, selection and prioritization. The formal definitions of these problems are presented in Section 2.2. The selected papers are listed in the Appendix. Fast abstracts and short papers have been excluded.
BACKGROUNDThis section introduces the basic concepts and definitions that form a nomenclature of regression testing and minimization, selection and prioritization techniques.
Definition 1 (Test Suite Minimization Problem)Given: A test suite, T , a set of test requirements {r 1 , . . .,r n }, that must be satisfied to provide the desired 'adequate' testing of the program, and subsets of T , T 1 , . . ., T n , one associated with each of the r i s such that any one of the test cases t j belonging to T i can be used to achieve requirement r i .Problem: Find a representative set, T , of test cases from T that satisfies all r i s.The testing criterion is satisfied when every test requirement in {r 1 , . . .,r n } is satisfied. A test requirement, r i , is satisfied by any test case, t j , that belongs to the T i , a subset of T . Therefore, the representative set of test cases is the hitting set of the T i s. Furthermore, in order to maximize the effect of minimization, T should be the minimal hitting set of the T i ...