Retest test selection for product-line regression testing of variants and versions of variants

Lity, Sascha; Nieke, Manuel; Thüm, Thomas; Schaefer, Ina

doi:10.1016/j.jss.2018.09.090

Cited by 24 publications

(27 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the literature we find mapping studies on regression testing and SPL engineering [11,35]. Existing works explore the three basic regression testing techniques [43]: minimization [40], selection [16,17,41,42], and prioritization [1,12,13] of test cases. These techniques are not excluding and can be combined [9].…”

Section: Related Workmentioning

confidence: 99%

“…6.7000 ± 0.466 ▲ 1.0000 ± 0.000 ⋆ 0.0035 ± 0.000 0.0041 ± 0.000 4.0000 ± 0.000 ▲ 1.6000 ± 0.747 ⋆ 0.0112 ± 0.000 0.0072 ± 0.004 16 5.0000 ± 0.000 ▲ 1.9112 ± 1.365 ⋆ 0.0570 ± 0.000 0.1201 ± 0.042 3.7000 ± 0.000 4.2333 ± 2.367 0.2263 ± 0.000 0.2236 ± 0.003 3.5000 ± 0.000 ▲ 1.9733 ± 0.481 ⋆ 0.2299 ± 0.000 0.2259 ± 0.002 fed-op-m --0.0000 ± 0.000 0.0000 ± 0.000 5.0000 ± 0.000 ▲ 1.0000 ± 0.000 ⋆ 0.0081 ± 0.000 0.0083 ± 0.000 5.0000 ± 0.000 ▲ 1.0000 ± 0.000 ⋆ 0.0081 ± 0.000 0.0083 ± 0.000 fed-und 3.6190 ± 0.427 ⋆ 3.9103 ± 0.228 △ 0.0181 ± 0.001 0.0165 ± 0.003 3.0769 ± 0.000 ▲ 2.4824 ± 0.501 ⋆ 0.0464 ± 0.000 0.0523 ± 0.001 3.5048 ± 0.090 ▲ 2.2713 ± 0.532 ⋆ 0.0497 ± 0.000 0.0528 ± 0.001 freebsd 8.6667 ± 0.000 ▲ 1.0000 ± 0.000 ⋆ 0.0281 ± 0.000 0.0282 ± 0.000 5.3333 ± 0.000 ▲ 1.0000 ± 0.000 ⋆ 0.0281 ± 0.000 0.0282 ± 0.000 5.3333 ± 0.000 ▲ 1.0000 ± 0.000 ⋆ 0.0281 ± 0.000 0.0282 ± 0.000 mingw32…”

Section: Rq3: Comparing Vts and Wts Strategiesunclassified

See 1 more Smart Citation

Learning-based prioritization of test cases in continuous integration of highly-configurable software

Lima

Mendonça

Vergílio

et al. 2020

Proceedings of the 24th ACM Conference on Systems and Software Product Line: Volume a - Volume A

View full text Add to dashboard Cite

Continuous Integration (CI) is a practice widely adopted in the industry to allow frequent integration of code changes. During the CI process, many test cases are executed multiple times a day, subject to time constraints. In this scenario, a learning-based approach, named COLEMAN, has been successfully applied. COLEMAN allows earlier execution of the most promising test cases to reveal faults. This approach considers CI particularities such as time budget and volatility of test cases, related to the fact that test cases can be added/removed along the CI cycles. In the CI of Highly Configuration System (HCS), many product variants must be tested, each one with different configuration options, but having test cases that are common to or reused from other variants. In this context, we found, by analogy, another particularity, the volatility of variants, that is, some variants can be included/discontinued along CI cycles. Considering this context, this work introduces two strategies for the application of COLEMAN in the CI of HCS: the Variant Test Set Strategy (VTS) that relies on the test set specific for each variant, and the Whole Test Set Strategy (WST) that prioritizes the test set composed by the union of the test cases of all variants. Both strategies are evaluated in a real-world HCS, considering three test budgets. The results show that the proposed strategies are applicable regarding the time spent for prioritization. They perform similarly regarding early fault detection, but WTS better mitigates the problem of beginning without knowledge, and is more suitable when a new variant to be tested is added. CCS CONCEPTS • General and reference → Experimentation; • Software and its engineering → Software product lines; Software testing and debugging.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Rq3: Comparing Vts and Wts Strategiesunclassified

Learning-based prioritization of test cases in continuous integration of highly-configurable software

Lima

Mendonça

Vergílio

et al. 2020

Proceedings of the 24th ACM Conference on Systems and Software Product Line: Volume a - Volume A

View full text Add to dashboard Cite

show abstract

“…Lity et al [4] proposed a model-based RTS method for product lines. However, CPLTE cannot be used with the RTS method of Jung et al [36] because their method requires the state machines of each product of a product family and our subject product lines do not have them.…”

Section: Test Selection Strategymentioning

confidence: 99%

“…The RTS methods [4,5,36,40] for SPL have been proposed to reduce redundancy of test cases for retest of a modified product family. Jung et al [36] propose an automated code-based RTS method, which reduces unnecessary testing cost by reusing the testing artifacts of one product for the other products and by avoiding the in-depth analysis of source code and test cases.…”

Section: Studies On Reducing Redundant Test Cases For Splmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Regression Testing of Software Product Lines by Reducing Redundant Test Executions

2020

View full text Add to dashboard Cite

Regression testing for software product lines (SPLs) is challenging because it must ensure that all the products of a product family work correctly whenever changes are made. One approach to reducing the cost of regression testing is the regression test selection (RTS), which selects a subset of regression test cases. However, even when RTS is applied, SPL regression testing can still be expensive because, in the product line context, each test case can be executed on more than one product that reuses the test case, which would typically result in a large number of test executions. A promising direction is to eliminate redundant test executions of test cases. We propose a method that, given a test case, identifies a set of products, on which the test case will cover the same sequence of source code statements and produce the same testing results, and then excludes these products from products to apply the test case to. The evaluation results showed that when the full selection approach and the approach of repetitively applying an RTS method for a single software system are used for test selection, our method reduced, respectively, 59.3% and 40.0% of the numbers of test executions of the approaches.

show abstract

Cost-effective learning-based strategies for test case prioritization in continuous integration of highly-configurable software

et al. 2022

View full text Add to dashboard Cite

Retest test selection for product-line regression testing of variants and versions of variants

Cited by 24 publications

References 26 publications

Learning-based prioritization of test cases in continuous integration of highly-configurable software

Learning-based prioritization of test cases in continuous integration of highly-configurable software

Efficient Regression Testing of Software Product Lines by Reducing Redundant Test Executions

Cost-effective learning-based strategies for test case prioritization in continuous integration of highly-configurable software

Contact Info

Product

Resources

About