Using semi-supervised learning for predicting metamorphic relations

Hardin, Bonnie; Kanewala, Upulee

doi:10.1145/3193977.3193985

Cited by 22 publications

(24 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feature extraction: Function cfg is the feature extracted by the majority of the studies [43,44,130,150,151] in the sub-category. They aimed to train models for predicting the metamorphic relations for testing.…”

Section: Test Data and Test Cases Generationmentioning

confidence: 99%

A Survey on Machine Learning Techniques for Source Code Analysis

Sharma¹,

Kechagia²,

Georgiou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Context:The advancements in machine learning techniques have encouraged researchers to apply these techniques to a myriad of software engineering tasks that use source code analysis such as testing and vulnerabilities detection. A large number of studies poses challenges to the community to understand the current landscape. Objective: We aim to summarize the current knowledge in the area of applied machine learning for source code analysis. Method: We investigate studies belonging to twelve categories of software engineering tasks and corresponding machine learning techniques, tools, and datasets that have been applied to solve them. To do so, we carried out an extensive literature search and identified 364 primary studies published between 2002 and 2021. We summarize our observations and findings with the help of the identified studies. Results: Our findings suggest that the usage of machine learning techniques for source code analysis tasks is consistently increasing. We synthesize commonly used steps and the overall workflow for each task, and summarize the employed machine learning techniques. Additionally, we collate a comprehensive list of available datasets and tools useable in this context. Finally, we summarize the perceived challenges in this area that include availability of standard datasets, reproducibility and replicability, and hardware resources. CCS Concepts: • Software and its engineering → Software libraries and repositories; Software maintenance tools; Software post-development issues; Maintaining software; • Computing methodologies → Machine learning.

show abstract

Section: Test Data and Test Cases Generationmentioning

confidence: 99%

A Survey on Machine Learning Techniques for Source Code Analysis

Sharma¹,

Kechagia²,

Georgiou³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The predicted behavior can vary in its level of abstraction, from a concrete output to a broad class of outputgenerally leaning more towards the abstract, given the challenges of making specific predictions for complex systems. • Metamorphic Relations: A metamorphic relation is a necessary property of a function of the software, relating input to the output produced [14]. For example, a metamorphic relation for a function is ( ) = ( − ).…”

Section: Test Oracle Types and Motivationmentioning

confidence: 99%

“…Metamorphic Relations: Six approaches generate metamorphic relations-necessary properties of a function that explain how a particular input links to its corresponding output [14]. Such relations allow us to infer expected results for different input values to a function, and violations of such properties identify potential faults.…”

Section: Application Of Machine Learningmentioning

confidence: 99%

See 1 more Smart Citation

Using machine learning to generate test oracles: a systematic literature review

Fontes

2021

Proceedings of the 1st International Workshop on Test Oracles

View full text Add to dashboard Cite

Machine learning may enable the automated generation of test oracles. We have characterized emerging research in this area through a systematic literature review examining oracle types, researcher goals, the ML techniques applied, how the generation process was assessed, and the open research challenges in this emerging field.Based on a sample of 22 relevant studies, we observed that ML algorithms generated test verdict, metamorphic relation, and-most commonly-expected output oracles. Almost all studies employ a supervised or semi-supervised approach, trained on labeled system executions or code metadata-including neural networks, support vector machines, adaptive boosting, and decision trees. Oracles are evaluated using the mutation score, correct classifications, accuracy, and ROC. Work-to-date show great promise, but there are significant open challenges regarding the requirements imposed on training data, the complexity of modeled functions, the ML algorithms employedand how they are applied-the benchmarks used by researchers, and replicability of the studies. We hope that our findings will serve as a roadmap and inspiration for researchers in this field. CCS CONCEPTS• Software and its engineering → Software verification and validation; • Computing methodologies → Machine learning.

show abstract

“…There has been a significant amount of work done on applying machine learning to solve software testing problems compared to testing machine learning applications [19]. For example, machine learning has been used to predict likely MRs for a given programs [20]- [23].…”

Section: Related Workmentioning

confidence: 99%

Fault Detection Effectiveness of Metamorphic Relations Developed for Testing Supervised Classifiers

Saha

Kanewala

2019

2019 IEEE International Conference on Artificial Intelligence Testing (AITest)

Self Cite

View full text Add to dashboard Cite

In machine learning, supervised classifiers are used to obtain predictions for unlabeled data by inferring prediction functions using labeled data. Supervised classifiers are widely applied in domains such as computational biology, computational physics and healthcare to make critical decisions. However, it is often hard to test supervised classifiers since the expected answers are unknown. This is commonly known as the oracle problem and metamorphic testing (MT) has been used to test such programs. In MT, metamorphic relations (MRs) are developed from intrinsic characteristics of the software under test (SUT). These MRs are used to generate test data and to verify the correctness of the test results without the presence of a test oracle. Effectiveness of MT heavily depends on the MRs used for testing. In this paper we have conducted an extensive empirical study to evaluate the fault detection effectiveness of MRs that have been used in multiple previous studies to test supervised classifiers. Our study uses a total of 709 reachable mutants generated by multiple mutation engines and uses data sets with varying characteristics to test the SUT. Our results reveal that only 14.8% of these mutants are detected using the MRs and that the fault detection effectiveness of these MRs do not scale with the increased number of mutants when compared to what was reported in previous studies.

show abstract

Using semi-supervised learning for predicting metamorphic relations

Cited by 22 publications

References 7 publications

A Survey on Machine Learning Techniques for Source Code Analysis

A Survey on Machine Learning Techniques for Source Code Analysis

Using machine learning to generate test oracles: a systematic literature review

Fault Detection Effectiveness of Metamorphic Relations Developed for Testing Supervised Classifiers

Contact Info

Product

Resources

About