A Unified Test Case Prioritization Approach

Abstract

Test case prioritization techniques attempt to reorder test cases in a manner that increases the rate at which faults are detected during regression testing. Coverage-based test case prioritization techniques typically use one of two overall strategies: a total strategy or an additional strategy . These strategies prioritize test cases based on the total number of code (or code-related) elements covered per test case and the number of additional (not yet covered) code (or code-related) elements covered per test case, respectively. In this article, we present a unified test case prioritization approach that encompasses both the total and additional strategies. Our unified test case prioritization approach includes two models ( basic and extended ) by which a spectrum of test case prioritization techniques ranging from a purely total to a purely additional technique can be defined by specifying the value of a parameter referred to as the f p value. To evaluate our approach, we performed an empirical study on 28 Java objects and 40 C objects, considering the impact of three internal factors (model type, choice of f p value, and coverage type) and three external factors (coverage granularity, test case granularity, and programming/testing paradigm), all of which can be manipulated by our approach. Our results demonstrate that a wide range of techniques derived from our basic and extended models with uniform f p values can outperform purely total techniques and are competitive with purely additional techniques. Considering the influence of each internal and external factor studied, the results demonstrate that various values of each factor have nontrivial influence on test case prioritization techniques.

Coverage-Based Clustering and Scheduling Approach for Test Case Prioritization

“…Jiang et al [14] propose adaptive random prioritization (ART). Hao et al [8] propose two models that unify the total and additional algorithms. Their empirical results prove that the additional greedy algorithm is the most effective TCP approach in terms of fault detection capability.…”

“…In this paper, we utilize two types of information, code coverage and fault detection rates of test cases which are both helpful to improve the effectiveness of TCP techniques, to prioritize test cases. The code coverage information is one of the most widely used information when implementing test case prioritization techniques [6], [8]- [10], [14]. In our work, first, merging test cases into clusters is based on the similarities of their code coverage; second, we also use code coverage information to help prioritization when giving initial order of test cases in clusters and selecting test cases for execution.…”

“…As mentioned in Sect. 2, the first three approaches are popularly used to benchmark other TCP approaches, and among them, additional greedy algorithm is widely considered as the most effective TCP approach to quickly expose faults from a program in terms of fault detection capacity [8]. TCP algorithm in [28] uses a clustering algorithm with three kinds of historical data.…”

“…Research shows that TCP can help tester improve the rate of fault detection and the developers can start debugging work early [6], [7]. To date, various prioritization techniques have been proposed and empirically studied [3], [6]- [8]. These techniques analyze source codes and historical execution data to obtain useful information, such as code coverage of test cases, code complexity, test costs, fault severities, in the form of alone or mix and then prioritize these test cases [6], [9]- [11].…”

Prioritizing MCDC test cases by spectral analysis of Boolean functions

“…Among them, the aim of test case prioritization is to rank the test cases to reveal faults earlier in order to reduce the cost of testing. There are numerous test case prioritization methods proposed in the literature [2,3,4,5,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55].…”