Abstract:Test case prioritization (TCP), which aims to find the optimal test case execution sequences for specific testing objects, has been widely used in regression testing. A wide variety of search methodologies and algorithms have been proposed to optimize test case execution sequences, namely, search‐based TCP. However, different algorithms perform differently and have different implementation costs and specific situations where an algorithm usually performs with high effectiveness and efficiency. When facing a ne… Show more
“…This study reported an average APSC of 96.34% for statement coverage and 738.87 as an average of the EET when 100% statement coverage was achieved for several iterations compared to 18734.4 seconds of the average execution time with reduction of around 96% of the actual time. This study argues that the execution time for this framework is always shorter than the multi-objective PSO algorithm [7].…”
Section: B Multi-objective Frameworkmentioning
confidence: 92%
“…Bian, Li, Gao and Zhao [7] proposed a concrete hyper-heuristic framework for prioritizing test cases. The main purpose of this framework is to select, among several algorithms in the repository, the appropriate algorithm for different TCP scenarios.…”
Section: B Multi-objective Frameworkmentioning
confidence: 99%
“…Furthermore, F5 was evaluated using the APFD C metric, which is derived from the APFD, but incorporates the aspects of cost and fault severity and is calculated as follows [1], [25 Where t j is the cost of test case, and f i fault severity. Next, F9 was evaluated using a weighted average percentage of statement coverage (APSC), which is also derived from the APFD for scaling statement coverage and is calculated using the following equation [7], [33]:…”
Section: A Evaluation Of Tcp Frameworkmentioning
confidence: 99%
“…A hybrid method combines TCP with other selection methods. In comparison to the selection and hybrid methods, test prioritization is considered as the most effective method in regression testing due to the higher possibility of finding hidden errors, as no test case is removed [7], [8].…”
Regression testing is a necessary maintenance activity in the software industry where modified software programs are revalidated to make sure that changes do not adversely affect their behavior. Test case prioritization (TCP) is one of the most effective methods in regression testing whereby test cases are rescheduled in an appropriate order for execution to increase test effectiveness in meeting some performance goals such as increasing the rate of fault detection. This paper explores efforts that have been carried out in relation to TCP frameworks. Through the review of related literature, ten existing frameworks were identified, classified and reviewed whereby two are Bayesian network-based, five are multi-objective, while the rest are varied in terms of aspects and purposes. Accordingly, this study analyzes those frameworks based on their proposed year, TCP factors, number of test cases used, evaluations metric and criteria as well as experimental subjects. The results showed that the stated frameworks are not integrated with nature-inspired algorithms as enhancing optimization techniques while several others were insufficiently evaluated according to stated evaluation criteria and metrics for the effective and practical testing process. There is also a scarcity of frameworks that focus on regression test efficiency. This study indicates the need for further research into the topic to enhance TCP frameworks that focus on several directions for practical considerations in this field such as evaluation issues, specific knowledge dependency, and objective deviation. At the end of this study, several future directions such as nature-inspired algorithms assistance are proposed, and a number of limitations are identified and highlighted.
“…This study reported an average APSC of 96.34% for statement coverage and 738.87 as an average of the EET when 100% statement coverage was achieved for several iterations compared to 18734.4 seconds of the average execution time with reduction of around 96% of the actual time. This study argues that the execution time for this framework is always shorter than the multi-objective PSO algorithm [7].…”
Section: B Multi-objective Frameworkmentioning
confidence: 92%
“…Bian, Li, Gao and Zhao [7] proposed a concrete hyper-heuristic framework for prioritizing test cases. The main purpose of this framework is to select, among several algorithms in the repository, the appropriate algorithm for different TCP scenarios.…”
Section: B Multi-objective Frameworkmentioning
confidence: 99%
“…Furthermore, F5 was evaluated using the APFD C metric, which is derived from the APFD, but incorporates the aspects of cost and fault severity and is calculated as follows [1], [25 Where t j is the cost of test case, and f i fault severity. Next, F9 was evaluated using a weighted average percentage of statement coverage (APSC), which is also derived from the APFD for scaling statement coverage and is calculated using the following equation [7], [33]:…”
Section: A Evaluation Of Tcp Frameworkmentioning
confidence: 99%
“…A hybrid method combines TCP with other selection methods. In comparison to the selection and hybrid methods, test prioritization is considered as the most effective method in regression testing due to the higher possibility of finding hidden errors, as no test case is removed [7], [8].…”
Regression testing is a necessary maintenance activity in the software industry where modified software programs are revalidated to make sure that changes do not adversely affect their behavior. Test case prioritization (TCP) is one of the most effective methods in regression testing whereby test cases are rescheduled in an appropriate order for execution to increase test effectiveness in meeting some performance goals such as increasing the rate of fault detection. This paper explores efforts that have been carried out in relation to TCP frameworks. Through the review of related literature, ten existing frameworks were identified, classified and reviewed whereby two are Bayesian network-based, five are multi-objective, while the rest are varied in terms of aspects and purposes. Accordingly, this study analyzes those frameworks based on their proposed year, TCP factors, number of test cases used, evaluations metric and criteria as well as experimental subjects. The results showed that the stated frameworks are not integrated with nature-inspired algorithms as enhancing optimization techniques while several others were insufficiently evaluated according to stated evaluation criteria and metrics for the effective and practical testing process. There is also a scarcity of frameworks that focus on regression test efficiency. This study indicates the need for further research into the topic to enhance TCP frameworks that focus on several directions for practical considerations in this field such as evaluation issues, specific knowledge dependency, and objective deviation. At the end of this study, several future directions such as nature-inspired algorithms assistance are proposed, and a number of limitations are identified and highlighted.
“…TCP technique is proposed to define the execution order of test cases in a test suite according to given criteria. In other words, TCP is aimed to find an optimal text case execution to achieve specific testing objectives [116]. Maximum faults can be revealed from the execution of reordered test cases.…”
SummaryHeuristic algorithms are widely used to solve multi‐objective test case prioritization (MOTCP) problems. However, they perform differently for different test scenarios, which conducts difficulty in applying a suitable algorithm for new test requests in the industry. A concrete hyper‐heuristic framework for MOTCP (HH‐MOTCP) is proposed for addressing this problem. It mainly has two parts: low‐level encapsulating various algorithms and hyper‐level including an evaluation and selection mechanism that dynamically selects low‐level algorithms. This framework performs good but still difficult to keep in the best three. If the evaluation mechanism can more accurately analyse the current results, it will help the selection strategy to find more conducive algorithms for evolution. Meanwhile, if the selection strategy can find a more suitable algorithm for the next generation, the performance of the HH‐MOTCP framework will be better. In this paper, we first propose new strategies for evaluating the current generation results, then perform an extensive study on the selection strategies which decide the heuristic algorithm for the next generation. Experimental results show that the new evaluation and selection strategies proposed in this paper can make the HH‐MOTCP framework more effective and efficient, which makes it almost the best two except for one test object and ahead in about 40% of all test objects.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.