Improving Regression Testing Transparency and Efficiency with History-Based Prioritization -- An Industrial Case Study

Engström, Emelie; Runeson, Per; Ljung, Andreas

doi:10.1109/icst.2011.27

Cited by 44 publications

(51 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Selection can be based on properties such as code coverage [14], most likely or expected locations of faults [16], topic coverage [8] or historic test execution data [3]. Prioritization can be viewed as a type of selection, as the first tests in an ordered suite may be the only ones run.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors

Strandberg

Sundmark

Afzal

et al. 2018

EasyChair Preprints

View full text Add to dashboard Cite

Abstract-We propose a new method of determining an effective ordering of regression test cases, and describe its implementation as an automated tool called SuiteBuilder developed by Westermo Research and Development AB. The tool generates an efficient order to run the cases in an existing test suite by using expected or observed test duration and combining priorities of multiple factors associated with test cases, including previous fault detection success, interval since last executed, and modifications to the code tested. The method and tool were developed to address problems in the traditional process of regression testing, such as lack of time to run a complete regression suite, failure to detect bugs in time, and tests that are repeatedly omitted. The tool has been integrated into the existing nightly test framework for Westermo software that runs on largescale data communication systems. In experimental evaluation of the tool, we found significant improvement in regression testing results. The re-ordered test suites finish within the available time, the majority of fault-detecting test cases are located in the first third of the suite, no important test case is omitted, and the necessity for manual work on the suites is greatly reduced.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Prioritization can be viewed as a type of selection, as the first tests in an ordered suite may be the only ones run. Overviews of selection and prioritization techniques appear in [21], [3], and [4].…”

Section: Related Workmentioning

confidence: 99%

Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors

Strandberg

Sundmark

Afzal

et al. 2018

EasyChair Preprints

View full text Add to dashboard Cite

show abstract

“…In order to analyze the economic feasibility of our approach, we carried out a case study at Bombardier Transportation (BT) in Sweden, inspired by the guidelines of Runeson and Höst [28] and specifically the way guidelines are followed in the paper by Engström et al [4]. We investigated the software/hardware integration testing process for the train control management subsystem (TCMS) in the Trenitalia Frecciarossa 1000, a non-articulated high-speed trainset.…”

Section: Case Studymentioning

confidence: 99%

“…Numerous techniques for test case selection and prioritization have been proposed in the last decade [2], [3], [4]. Most of the proposed techniques for ordering test cases are offline, meaning that the order is decided before execution while the current execution results do not play a part in prioritizing or selecting test cases to execute.…”

Section: Introductionmentioning

confidence: 99%

Cost-Benefit Analysis of Using Dependency Knowledge at Integration Testing

Tahvili

Bohlin²,

Saadatmand

et al. 2016

Product-Focused Software Process Improvement

View full text Add to dashboard Cite

Abstract. In software system development, testing can take considerable time and resources, and there are numerous examples in the literature of how to improve the testing process. In particular, methods for selection and prioritization of test cases can play a critical role in efficient use of testing resources. This paper focuses on the problem of selection and ordering of integration-level test cases. Integration testing is performed to evaluate the correctness of several units in composition. Further, for reasons of both effectiveness and safety, many embedded systems are still tested manually. To this end, we propose a process, supported by an online decision support system, for ordering and selection of test cases based on the test result of previously executed test cases. To analyze the economic efficiency of such a system, a customized return on investment (ROI) metric tailored for system integration testing is introduced. Using data collected from the development process of a large-scale safety-critical embedded system, we perform Monte Carlo simulations to evaluate the expected ROI of three variants of the proposed new process. The results show that our proposed decision support system is beneficial in terms of ROI at system integration testing and thus qualifies as an important element in improving the integration testing process.

show abstract

“…[8]. Prioritization criteria may be based on test case history [9], coverage [10], fault severity [11], customer requirements [12] or costs [13].…”

mentioning

confidence: 99%

On the Applicability of Soft Computing Techniques in Regression Testing

Khanna¹,

Khanna²

2017

IJCA

View full text Add to dashboard Cite

Regression testing is required whenever changes are incorporated in software. Regression testing re-executes the test cases to verify that changes made during maintainance phase have not caused any unintended effects. Test case prioritization technique re-orders the test cases in such a way that important test cases are executed within the resources. The work reviews different test case prioritization techniques. The work also presents computational intelligence based test case prioritization techniques. The work reviews test case prioritization based on Neural Networks, Genetic Algorithms and Fuzzy Logic.

show abstract

Improving Regression Testing Transparency and Efficiency with History-Based Prioritization -- An Industrial Case Study

Cited by 44 publications

References 23 publications

Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors

Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors

Cost-Benefit Analysis of Using Dependency Knowledge at Integration Testing

On the Applicability of Soft Computing Techniques in Regression Testing

Contact Info

Product

Resources

About