Automated metamorphic testing

Gotlieb, Arnaud; Botella, Bernard

doi:10.1109/cmpsac.2003.1245319

Cited by 67 publications

(42 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each executed mutant, we ran our test data generator until finding a test case that kills it or until a timeout of 600 seconds was exceeded. We chose this value for the timeout because it had proved to be effective in similar experiments with mutants and automated metamorphic testing [14]. 4.…”

Section: Methodsmentioning

confidence: 99%

Mutation testing on an object-oriented framework: An experience report

Segura

Hierons

Benavides

et al. 2011

Information and Software Technology

View full text Add to dashboard Cite

Context: The increasing presence of Object-Oriented (OO) programs in industrial systems is progressively drawing the attention of mutation researchers toward this paradigm. However, while the number of research contributions in this topic is plentiful, the number of empirical results is still marginal and mostly provided by researchers rather than practitioners. Objective: This article reports our experience using mutation testing to measure the effectiveness of an automated test data generator from a user perspective. Method: In our study, we applied both traditional and class-level mutation operators to FaMa, an open source Java framework currently being used for research and commercial purposes. We also compared and contrasted our results with the data obtained from some motivating faults found in the literature and two real tools for the analysis of feature models, FaMa and SPLOT. Results: Our results are summarized in a number of lessons learned supporting previous isolated results as well as new findings that hopefully will motivate further research in the field. Conclusion: We conclude that mutation testing is an effective and affordable technique to measure the effectiveness of test mechanisms in OO systems. We found, however, several practical limitations in current tool support that should be addressed to facilitate the work of testers. We also missed specific techniques and tools to apply mutation testing at the system level.

show abstract

Section: Methodsmentioning

confidence: 99%

Mutation testing on an object-oriented framework: An experience report

Segura

Hierons

Benavides

et al. 2011

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…Gotlieb and Botella [16] developed an automated framework for a class of metamorphic relations. The framework translates a specification into a constraint logic programming (CLP) program.…”

Section: Metamorphic Testing Proceduresmentioning

confidence: 99%

“…They [15] further investigated the integration of metamorphic testing with fault-based testing and global symbolic evaluation. Gotlieb and Botella [16] developed an automated framework to check against a class of metamorphic relations. Chan and colleagues applied metamorphic approach to the unit testing [17] and integration testing [9] of context-sensitive middleware-based applications.…”

Section: Introductionmentioning

confidence: 99%

Experimental Study to Compare the Use of Metamorphic Testing and Assertion Checking

Zhang¹,

Chan²,

Tse³

et al. 2009

Journal of Software

View full text Add to dashboard Cite

A test oracle in software testing is a mechanism for checking whether the program under test behaves correctly for any execution. In some practical situations, oracles can be unavailable or too expensive to apply.Metamorphic testing (MT) was proposed to alleviate this problem so that software can be delivered under the time-to-market pressure. However, the effectiveness of MT has not been studied adequately. This paper conducts a controlled experiment to investigate the cost effectiveness of using MT. The fault detection capability and time cost of MT are compared with the standard assertion checking method. Our results show that MT has potentials to detect more faults than the assertion checking method. The experimental results also show a trade-off between the two testing methods: MT can be less efficient but more effective, and can be defined at a coarser level of granularity th an the assertion checking method.

show abstract

“…In that case, mutational metamorphic testing work like fuzz testing, hence the name of the testing tool JFuzz presented in this paper. In contrast, a testing tool that support the general metamorphic testing method has to rely on constraint solver to generate test cases that satisfy the input constraints; see for example, [15].…”

Section: Resultsmentioning

confidence: 99%