An approach to design test oracle for aspect oriented software systems using soft computing approach

Singhal, Abhishek; Bansal, Abhay; Kumar, Avadhesh

doi:10.1007/s13198-015-0402-2

Cited by 7 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[96] model a function that judges whether an integer is prime-a binary classification problem. [85,90,91,92,93] also generate oracles for applications with enumerated output.…”

Section: Test Oracle Generationmentioning

confidence: 99%

See 1 more Smart Citation

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Fontes¹,

Gay²

2022

Preprint

View full text Add to dashboard Cite

Context: Machine learning (ML) may enable effective automated test generation.Objectives: We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges.Methods: We perform a systematic literature review on a sample of 97 publications.Results: ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property-based, and expected output oracles. Supervised learning-often based on neural networks-and reinforcement learning-often based on Q-learning-are common, and some publications also employ unsupervised or semi-supervised learning. (Semi-/Un-)Supervised approaches are evaluated using both traditional testing metrics and ML-related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. Conclusion:Work-to-date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed-and how they are applied-benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.

show abstract

“…[96] model a function that judges whether an integer is prime-a binary classification problem. [85,90,91,92,93] also generate oracles for applications with enumerated output.…”

Section: Test Oracle Generationmentioning

confidence: 99%

“…The majority of approaches use of some form of NN [83,87,13,88,89,90,91,92,94,95,96]. [85] technique where labeled and unlabeled training data are used, and the algorithm propagates labels to similar, unlabeled data to reduce the quantity of required labeling.…”

Section: Test Oracle Generationmentioning

confidence: 99%

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Fontes¹,

Gay²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This is an even more straightforward applicationa two-class classification problem. Shahamiri et al [32,34] generate oracles for a car insurance application, while Singhal et al [35] and Vanmali et al [37] generate oracles for a credit analysis at a bank. Ding et al [6] generate oracles for an image processing function that classifies a type of cell from image sections.…”

Section: Application Of Machine Learningmentioning

confidence: 99%

“…The other approaches follow a more traditional supervised, regressionbased learning process, and generally make use of different NNs. Four of the examined studies adopt a Backpropagation NN [1,16,35,38]. Three other studies employ the Multilayer Perceptron technique [32,34,39].…”

Section: Application Of Machine Learningmentioning

confidence: 99%

“…In terms of evaluation, five of the studies are focused on the accuracy of the oracle in a set of cases where the ground truth is known-measuring the percentage of correct classifications [1,16,31,41] or the accuracy of the model [35]. The remaining seven used the mutation score as the evaluation metric [6,24,32,34,[37][38][39].…”

Section: Application Of Machine Learningmentioning

confidence: 99%

See 1 more Smart Citation

Using machine learning to generate test oracles: a systematic literature review

Fontes

2021

Proceedings of the 1st International Workshop on Test Oracles

View full text Add to dashboard Cite

Machine learning may enable the automated generation of test oracles. We have characterized emerging research in this area through a systematic literature review examining oracle types, researcher goals, the ML techniques applied, how the generation process was assessed, and the open research challenges in this emerging field.Based on a sample of 22 relevant studies, we observed that ML algorithms generated test verdict, metamorphic relation, and-most commonly-expected output oracles. Almost all studies employ a supervised or semi-supervised approach, trained on labeled system executions or code metadata-including neural networks, support vector machines, adaptive boosting, and decision trees. Oracles are evaluated using the mutation score, correct classifications, accuracy, and ROC. Work-to-date show great promise, but there are significant open challenges regarding the requirements imposed on training data, the complexity of modeled functions, the ML algorithms employedand how they are applied-the benchmarks used by researchers, and replicability of the studies. We hope that our findings will serve as a roadmap and inspiration for researchers in this field. CCS CONCEPTS• Software and its engineering → Software verification and validation; • Computing methodologies → Machine learning.

show abstract

Machine learning‐based test oracles for performance testing of cyber‐physical systems: An industrial case study on elevators dispatching algorithms

Gartziandia

Arrieta

Ayerdi

et al. 2022

J Software Evolu Process

View full text Add to dashboard Cite

The software of systems of elevators needs constant maintenance to deal with new functionality, bug fixes, or legislation changes. To automatically validate the software of these systems, a typical approach in industry is to use regression oracles, which execute test inputs both in the software version under test and in a previous software version. However, these practices require a long test execution time and cannot be re‐used at different test phases. To deal with these issues, we propose Dispatching AlgoRIthm Oracle (DARIO), a test oracle that relies on regression machine‐learning algorithms to detect both functional and non‐functional problems of the system. The machine‐learning algorithms of this oracle are trained by using data from previously tested versions to predict reference functional and non‐functional performance values of the new versions. An empirical evaluation with an industrial case study demonstrates the feasibility of using our approach. A total of five regression learning algorithms were validated by using mutation testing techniques. For the context of functional bugs, the accuracy when predicting verdicts by DARIO ranged between 95% and 98%, across the different scenarios proposed. For the context of non‐functional bugs, were competitive too, having an accuracy when predicting verdicts by DARIO ranged between 83% and 87%.

show abstract

An approach to design test oracle for aspect oriented software systems using soft computing approach

Cited by 7 publications

References 9 publications

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Using machine learning to generate test oracles: a systematic literature review

Machine learning‐based test oracles for performance testing of cyber‐physical systems: An industrial case study on elevators dispatching algorithms

Contact Info

Product

Resources

About