Metamorphic relations to improve the test accuracy of Multi Precision Arithmetic software applications

Aruna, Chittineni; Prasad, R. Siva Ram

doi:10.1109/icacci.2014.6968586

Cited by 6 publications

(3 citation statements)

References 12 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Special test cases were unable to detect the fault, while metamorphic testing was effective at revealing it using a single metamorphic relation. Aruna and Prasad [117] presented several metamorphic relations for multiplication and division of multi-precision arithmetic software applications. The work was evaluated with four real-time mathematical projects and mutation analysis.…”

Section: Numerical Programsmentioning

confidence: 99%

A Survey on Metamorphic Testing

Segura

Fraser

Sánchez

et al. 2016

IIEEE Trans. Software Eng.

409

230

View full text Add to dashboard Cite

Abstract-A test oracle determines whether a test execution reveals a fault, often by comparing the observed program output to the expected output. This is not always practical, for example when a program's input-output relation is complex and difficult to capture formally. Metamorphic testing provides an alternative, where correctness is not determined by checking an individual concrete output, but by applying a transformation to a test input and observing how the program output "morphs" into a different one as a result. Since the introduction of such metamorphic relations in 1998, many contributions on metamorphic testing have been made, and the technique has seen successful applications in a variety of domains, ranging from web services to computer graphics. This article provides a comprehensive survey on metamorphic testing: It summarises the research results and application areas, and analyses common practice in empirical studies of metamorphic testing as well as the main open challenges.

show abstract

Section: Numerical Programsmentioning

confidence: 99%

A Survey on Metamorphic Testing

Segura

Fraser

Sánchez

et al. 2016

IIEEE Trans. Software Eng.

409

230

View full text Add to dashboard Cite

show abstract

“…In practice, MT frees engineers from implementing a specific oracle (e.g., test case assertions) for each test case. Considerable research has been devoted to developing MT approaches for various domains such as computer graphics (e.g., [19], [20], [21], [22]), simulation (e.g., [152], [153], [154]), Web services (e.g., [23], [24], [25]), embedded systems (e.g., [26], [27], [28], [29]), compilers (e.g., [156], [181]), variability and decision support (e.g., [158], [159], [160], [182]), bioinformatics (e.g., [161], [162], [183]), numerical programs (e.g., [163], [164]), and machine learning (e.g., [165], [166]). However, very little attention has been paid to its application in security testing [18].…”

Section: Metamorphic Testingmentioning

confidence: 99%

Metamorphic Testing for Web System Security

Chaleshtari¹,

Pastore

Göknil

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Security testing aims at verifying that the software meets its security properties. In modern Web systems, however, this often entails the verification of the outputs generated when exercising the system with a very large set of inputs. Full automation is thus required to lower costs and increase the effectiveness of security testing. Unfortunately, to achieve such automation, in addition to strategies for automatically deriving test inputs, we need to address the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior (e.g., the response to be received after a specific HTTP GET request). In this paper, we propose Metamorphic Security Testing for Web-interactions (MST-wi), a metamorphic testing approach that integrates test input generation strategies inspired by mutational fuzzing and alleviates the oracle problem in security testing. It enables engineers to specify metamorphic relations (MRs) that capture many security properties of Web systems. To facilitate the specification of such MRs, we provide a domain-specific language accompanied by an Eclipse editor. MST-wi automatically collects the input data and transforms the MRs into executable Java code to automatically perform security testing. It automatically tests Web systems to detect vulnerabilities based on the relations and collected data. We provide a catalog of 76 system-agnostic MRs to automate security testing in Web systems. It covers 39% of the OWASP security testing activities not automated by state-of-the-art techniques; further, our MRs can automatically discover 102 different types of vulnerabilities, which correspond to 45% of the vulnerabilities due to violations of security design principles according to the MITRE CWE database. We also define guidelines that enable test engineers to improve the testability of the system under test with respect to our approach. We evaluated MST-wi effectiveness and scalability with two well-known Web systems (i.e., Jenkins and Joomla). It automatically detected 85% of their vulnerabilities and showed a high specificity (99.81% of the generated inputs do not lead to a false positive); our findings include a new security vulnerability detected in Jenkins. Finally, our results demonstrate that the approach scale, thus enabling automated security testing overnight.

show abstract

“…Successful test cases of MT were reused to produce the next generation (follow-up) test cases, to reduce the burden of generating new test cases and to reveal the hidden bug information with exhaustive concentration. Analysis phase of this research identified that MT was suffering from evaluation of the complex Non-Testable program expressions [5] to reveal hidden bug information, considered as a serious limitation of MT explained in problem statement (section 2). As non-testable programs need to deal with rigid operations, frequently they were composed with complex expressions [6 and 12].…”

mentioning

confidence: 99%

A Naïve Diamond Interpolation Algorithm for Transparent Evaluation of Non-Testable Program Expressions in Metamorphic Testing

2016

IJCCIE

View full text Add to dashboard Cite

Abstract---Ubiquitous software Applications depends on NonTestable programs either directly or indirectly to achieve their computational tasks. Attribute relations based Metamorphic Testing emerged as a popular alternative to test the Non-Testable programs without using any test oracle. Our research analysis on metamorphic testing observed that, since beginning MT was suffering from evaluation of the complex Non-Testable program expressions to reveal hidden bug information. In traditional metamorphic testing strategy, it is even unable to find the erroneous part of complex expression which is more important for debugging and also caused to raise the exception. Best of my knowledge, none of the former scholars of MT was concerned on this serious issue, which effects on quality and scalability of metamorphic testing. In this paper we concentrated on above mentioned serious limitations of MT and proposed a Naïve Diamond Interpolation Algorithm for Transparent Evaluation of Non-Testable Program Expressions in Metamorphic Testing. This algorithm will be adopted as an integral part of metamorphic testing runtime environment to improvise the quality and scalability. Experimental results of this paper on various nontestable programs were compared against the traditional metamorphic testing. Results proven that the adoptions of Diamond Interpolation Algorithm dramatically increases quality and scalability statistics of metamorphic testing and also made the debugging job easier by identifying the erroneous part of nontestable program complex expression.

show abstract

Metamorphic relations to improve the test accuracy of Multi Precision Arithmetic software applications

Cited by 6 publications

References 12 publications

A Survey on Metamorphic Testing

A Survey on Metamorphic Testing

Metamorphic Testing for Web System Security

A Naïve Diamond Interpolation Algorithm for Transparent Evaluation of Non-Testable Program Expressions in Metamorphic Testing

Contact Info

Product

Resources

About