Test Case Design and Test Case Prioritization using Machine Learning

Sharna, Mayank Mohan; Agrawal, Akshat; Kumar, *B. Suresh

doi:10.35940/ijeat.a9762.109119

Cited by 6 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The graph can update itself after each cycle so that cycles would affect the order of test cases. Sharma and Agrawal built an information graph from UML and user story, then fed that graph into metaheuristic algorithms [14]. Kaur et al extracted the elements of UI in each test step, then used this data as input for traditional machine learning algorithms (SVM, decision trees, naive Bayes, etc.)…”

Section: Related Workmentioning

confidence: 99%

Ensemble approaches for Test Case Prioritization in UI testing

Cao¹,

Vu²,

Le³

et al. 2022

Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Test case prioritization, which focuses on ranking test cases, is an important activity in software engineering given a large number of test cases to be executed within a short period of time. Recent approaches use test execution history and test coverage as the key information for ranking prediction while reinforcement learning has the potential for improving the accuracy of prioritization. Still, each approach has its own advantages and limitations. This paper proposes a ensemble method to take advantages of several existing models by combining different them into a single one. We evaluate our ensemble models on the data sets, including sixteen projects. The results show that one of our proposed models outperforms all single models on 12 over 16 data sets.

show abstract

Section: Related Workmentioning

confidence: 99%

Ensemble approaches for Test Case Prioritization in UI testing

Cao¹,

Vu²,

Le³

et al. 2022

Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…The work in [26] uses regression trees to minimize test case redundancy in CI testing and improve the costeffectiveness of CI. Very recent work in [27] proposes machine learning for the generation and prioritization of test cases for a user interface. Their approach concludes that K-Nearest Neighbors performed well among the rest of the machine learning classification models on their collected historical data from user interface design.…”

Section: Related Workmentioning

confidence: 99%

DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing

Sharif¹,

Marijan²,

Liaaen³

2021

Preprint

View full text Add to dashboard Cite

Continuous integration testing is an important step in the modern software engineering life cycle. Test prioritization is a method that can improve the efficiency of continuous integration testing by selecting test cases that can detect faults in the early stage of each cycle. As continuous integration testing produces voluminous test execution data, test history is a commonly used artifact in test prioritization. However, existing test prioritization techniques for continuous integration either cannot handle large test history or are optimized for using a limited number of historical test cycles. We show that such a limitation can decrease fault detection effectiveness of prioritized test suites.This work introduces DeepOrder, a deep learning-based model that works on the basis of regression machine learning. DeepOrder ranks test cases based on the historical record of test executions from any number of previous test cycles. DeepOrder learns failed test cases based on multiple factors including the duration and execution status of test cases. We experimentally show that deep neural networks, as a simple regression model, can be efficiently used for test case prioritization in continuous integration testing. DeepOrder is evaluated with respect to timeeffectiveness and fault detection effectiveness in comparison with an industry practice and the state of the art approaches. The results show that DeepOrder outperforms the industry practice and state-of-the-art test prioritization approaches in terms of these two metrics.

show abstract

“…Rongqi et. al [8] surveyed ML-based TCP techniques and reported that existing ML-based papers investigated a wide variety of ML techniques including reinforcement learning [2], [23], [4], [3], [43], [44], [45], clustering [46], [47], [48], [49], [50], [51], [52], ranking models [23], [21], [31], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], and natural language processing [49], [21], [55], [63], [64]. However, each study only used a small number of features that are either easily collected or publicly published by existing work.…”

Section: Related Workmentioning

confidence: 99%

Scalable and Accurate Test Case Prioritization in Continuous Integration Contexts

Yaraghi,

Bagherzadeh,

Kahani

et al. 2021

Preprint

View full text Add to dashboard Cite

Continuous Integration (CI) requires efficient regression testing to ensure software quality without significantly delaying its CI builds. This warrants the need for techniques to reduce regression testing time, such as Test Case Prioritization (TCP) techniques that prioritize the execution of test cases to detect faults as early as possible. Many recent TCP studies employ various Machine Learning (ML) techniques to deal with the dynamic and complex nature of CI. However, most of them use a limited number of features for training ML models and evaluate the models on subjects for which the application of TCP makes little practical sense, due to their small regression testing time and low number of failed builds. In this work, we first define, at a conceptual level, a data model that captures data sources and their relations in a typical CI environment. Second, based on this data model, we define a comprehensive set of features that covers all features previously used by related studies. Third, we develop methods and tools to collect the defined features for 25 open-source software systems with enough failed builds and whose regression testing takes at least five minutes. Fourth, relying on the collected dataset containing a comprehensive feature set, we answer four research questions concerning data collection time, the accuracy of ML-based TCP, the impact of the features on accuracy, the decay of ML-based TCP models over time, and the trade-off between data collection time and the accuracy of ML-based TCP techniques.

show abstract

Test Case Design and Test Case Prioritization using Machine Learning

Cited by 6 publications

References 13 publications

Ensemble approaches for Test Case Prioritization in UI testing

Ensemble approaches for Test Case Prioritization in UI testing

DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing

Scalable and Accurate Test Case Prioritization in Continuous Integration Contexts

Contact Info

Product

Resources

About