A machine learning approach to generate test oracles

Braga, Ronyerison; Neto, Pedro; Rabelo, Ricardo A. L.; Santiago, José; Souza, Matheus

doi:10.1145/3266237.3266273

Cited by 24 publications

(16 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accuracy of the proposed oracle was up to 98.26%, and the oracle detected up to 97.7% of the injected faults. Braga et al [62] uses historical usage data from an application that goes through a Knowledge Discovery in Database step and is then used for training (using AdaBoostM1 and Incremental Reduced Error Pruning (IREP) ) to generate an oracle suitable for the application under test. Chan et al [63] developed an approach that trains a classifier using a reference model of the SUT.…”

Section: Last and Friedmanmentioning

confidence: 99%

Artificial Intelligence in Software Testing : Impact, Problems, Challenges and Prospect

Khaliq¹,

Farooq²,

Khan³

2022

Preprint

View full text Add to dashboard Cite

Artificial Intelligence (AI) is making a significant impact in multiple areas like medical, military, industrial, domestic, law, arts as AI is capable to perform several roles such as managing smart factories, driving autonomous vehicles, creating accurate weather forecasts, detecting cancer and personal assistants, etc. Software testing is the process of putting the software to test for some abnormal behaviour of the software. Software testing is a tedious, laborious and most time-consuming process. Automation tools have been developed that help to automate some activities of the testing process to enhance quality and timely delivery. Over time with the inclusion of continuous integration and continuous delivery (CI/CD) pipeline, automation tools are becoming less effective. The testing community is turning to AI to fill the gap as AI is able to check the code for bugs and errors without any human intervention and in a much faster way than humans. In this study, we aim to recognize the impact of AI technologies on various software testing activities or facets in the STLC. Further, the study aims to recognize and explain some of the biggest challenges software testers face while applying AI to testing. The paper also proposes some key contributions of AI in the future to the domain of software testing.

show abstract

Section: Last and Friedmanmentioning

confidence: 99%

Artificial Intelligence in Software Testing : Impact, Problems, Challenges and Prospect

Khaliq¹,

Farooq²,

Khan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In essence, a BP NN "learns" by reducing error rates by tuning the weights in each neuron after computing the error, making the model more stable. Braga et al [5] use a classifier based on adaptive boosting.…”

Section: Application Of Machine Learningmentioning

confidence: 99%

“…Braga et al [5] gather usage data from a shopping website by inserting several specific capture components into the site. The data then goes through a preprocessing step and then is finally used for training the ML.…”

Section: Application Of Machine Learningmentioning

confidence: 99%

See 1 more Smart Citation

Using machine learning to generate test oracles: a systematic literature review

Fontes

2021

Proceedings of the 1st International Workshop on Test Oracles

View full text Add to dashboard Cite

Machine learning may enable the automated generation of test oracles. We have characterized emerging research in this area through a systematic literature review examining oracle types, researcher goals, the ML techniques applied, how the generation process was assessed, and the open research challenges in this emerging field.Based on a sample of 22 relevant studies, we observed that ML algorithms generated test verdict, metamorphic relation, and-most commonly-expected output oracles. Almost all studies employ a supervised or semi-supervised approach, trained on labeled system executions or code metadata-including neural networks, support vector machines, adaptive boosting, and decision trees. Oracles are evaluated using the mutation score, correct classifications, accuracy, and ROC. Work-to-date show great promise, but there are significant open challenges regarding the requirements imposed on training data, the complexity of modeled functions, the ML algorithms employedand how they are applied-the benchmarks used by researchers, and replicability of the studies. We hope that our findings will serve as a roadmap and inspiration for researchers in this field. CCS CONCEPTS• Software and its engineering → Software verification and validation; • Computing methodologies → Machine learning.

show abstract

“…The model is trained so that the output is similar to the Software under Test (SUT). The trained model is used to predict and detect any wrong output [5]. This paper proposes an automated approach using two cascaded machine learning algorithms, first to detect the failures in the software and then to cluster similar failures so the engineer would not have to inspect all executions of the software.…”

Section: Introductionmentioning

confidence: 99%

Machine Learning in Failure Regions Detection and Parameters Analysis

Wahab¹,

Adawi²,

Khater³

2019

IJNDC

View full text Add to dashboard Cite

Testing automation is one of the challenges facing the software development industry, especially for large complex products. This paper proposes a mechanism called Multi Stage Failure Detector (MSFD) for automating black box testing using different machine learning algorithms. The input to MSFD is the tool's set of parameters and their value ranges. The parameter values are randomly sampled to produce a large number of parameter combinations that are fed into the software under test. Using neural networks, the resulting logs from the tool are classified into passing and failing logs and the failing logs are then clustered (using mean-shift clustering) into different failure types. MSFD provides visualization of the failures along with the responsible parameters. Experiments on and results for two real-world complex software products are provided, showing the ability of MSFD to detect all failures and cluster them into the correct failure types, thus reducing the analysis time of failures, improving coverage, and increasing productivity.

show abstract

A machine learning approach to generate test oracles

Cited by 24 publications

References 30 publications

Artificial Intelligence in Software Testing : Impact, Problems, Challenges and Prospect

Artificial Intelligence in Software Testing : Impact, Problems, Challenges and Prospect

Using machine learning to generate test oracles: a systematic literature review

Machine Learning in Failure Regions Detection and Parameters Analysis

Contact Info

Product

Resources

About