Metamorphic testing of a Monte Carlo modeling program

Ding, Junhua; Wu, Tong; Wu, Dianxiang; Lu, Jun Q.; Hu, Xinhua

doi:10.1145/1982595.1982597

Cited by 15 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a related project, Chen et al [37] proposed using metamorphic testing for the detection of faults in open queuing network modelling, a technique for planning the capacity of computer and communication systems. Ding et al [105] presented a case study on the detection of faults in a Monte Carlo modelling program for the simulation of photon propagation. Based on their previous work [43], the authors used code coverage criteria to guide the selection of effective metamorphic relations and the creation of test cases.…”

Section: Simulation and Modellingmentioning

confidence: 99%

A Survey on Metamorphic Testing

Segura

Fraser

Sánchez

et al. 2016

IIEEE Trans. Software Eng.

405

230

View full text Add to dashboard Cite

Abstract-A test oracle determines whether a test execution reveals a fault, often by comparing the observed program output to the expected output. This is not always practical, for example when a program's input-output relation is complex and difficult to capture formally. Metamorphic testing provides an alternative, where correctness is not determined by checking an individual concrete output, but by applying a transformation to a test input and observing how the program output "morphs" into a different one as a result. Since the introduction of such metamorphic relations in 1998, many contributions on metamorphic testing have been made, and the technique has seen successful applications in a variety of domains, ranging from web services to computer graphics. This article provides a comprehensive survey on metamorphic testing: It summarises the research results and application areas, and analyses common practice in empirical studies of metamorphic testing as well as the main open challenges.

show abstract

Section: Simulation and Modellingmentioning

confidence: 99%

A Survey on Metamorphic Testing

Segura

Fraser

Sánchez

et al. 2016

IIEEE Trans. Software Eng.

405

230

View full text Add to dashboard Cite

show abstract

“…Metamorphic testing has been used to test applications without oracles in different areas such as machine learning applications , healthcare simulations , Monte Carlo modelling , bioinformatics programmes , computer graphics , programmes with partial differential equations and variability analysis tools . Recently, metamorphic testing has been used to validate compilers and cloud models.…”

Section: Metamorphic Testingmentioning

confidence: 99%

“…Xie et al , used metamorphic testing to test machine learning applications. Metamorphic testing was used to test simulation software such as healthcare simulations and Monte Carlo modelling . Metamorphic testing has been used effectively in bioinformatics , computer graphics and for testing programmes with partial differential equations .…”

Section: Related Workmentioning

confidence: 99%

Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels

Kanewala

Bieman

Ben‐Hur

2015

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary Comprehensive, automated software testing requires an oracle to check whether the output produced by a test case matches the expected behaviour of the programme. But the challenges in creating suitable oracles limit the ability to perform automated testing in some programmes, and especially in scientific software. Metamorphic testing is a method for automating the testing process for programmes without test oracles. This technique operates by checking whether the programme behaves according to properties called metamorphic relations. A metamorphic relation describes the change in output when the input is changed in a prescribed way. Unfortunately, finding the metamorphic relations satisfied by a programme or function remains a labour‐intensive task, which is generally performed by a domain expert or a programmer. In this work, we propose a machine learning approach for predicting metamorphic relations that uses a graph‐based representation of a programme to represent control flow and data dependency information. In earlier work, we found that simple features derived from such graphs provide good performance. An analysis of the features used in this earlier work led us to explore the effectiveness of several representations of those graphs using the machine learning framework of graph kernels, which provide various ways of measuring similarity between graphs. Our results show that a graph kernel that evaluates the contribution of all paths in the graph has the best accuracy and that control flow information is more useful than data dependency information. The data used in this study are available for download at http://www.cs.colostate.edu/saxs/MRpred/functions.tar.gz to help researchers in further development of metamorphic relation prediction methods. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

“…Xie et al used metamorphic testing to test machine learning applications [7]. Metamorphic testing was used to test simulation software such as health care simulations [29] and Monte Carlo modeling [30]. Metamorphic testing has been used effectively in bioinformatics [31], computer graphics [32] and for testing programs with partial differential equations [33].…”

Section: Related Workmentioning

confidence: 99%

Using machine learning techniques to detect metamorphic relations for programs without test oracles

Kanewala

Bieman

2013

2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

Much software lacks test oracles, which limits automated testing. Metamorphic testing is one proposed method for automating the testing process for programs without test oracles. Unfortunately, finding the appropriate metamorphic relations required for use in metamorphic testing remains a labor intensive task, which is generally performed by a domain expert or a programmer. In this work we present a novel approach for automatically predicting metamorphic relations using machine learning techniques. Our approach uses a set of features developed using the control flow graph of a function for predicting likely metamorphic relations. We show the effectiveness of our method using a set of real world functions often used in scientific applications.

show abstract

Metamorphic testing of a Monte Carlo modeling program

Cited by 15 publications

References 19 publications

A Survey on Metamorphic Testing

A Survey on Metamorphic Testing

Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels

Using machine learning techniques to detect metamorphic relations for programs without test oracles

Contact Info

Product

Resources

About