Nequivack: Assessing Mutation Score Confidence

Holling, Dominik; Bănescu, Sebastian; Probst, Marco; Petrovska, Ana; Pretschner, Alexander

doi:10.1109/icstw.2016.29

Cited by 12 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A different attempt to solve the same problem is based on identifying killable mutants. This has been attempted using (static) symbolic execution [149,156]. Such attempts aim at executing mutants symbolically in order to identify whether these can be killable with symbolic input data.…”

Section: Identifying Equivalent Mutantsmentioning

confidence: 99%

“…Other attempts are due to Papadakis and Malevris [177] and Holling et al [156] who used out of the box symbolic execution engines (JPF-SE [178] and KLEE [179] respectively) to generate mutation-based test cases. These approaches instrument the original program with mutant killing conditions that the symbolic execution engine is asked to cover (transforms the mutant killing problem to code reachability problem).…”

Section: Static Constraint-based Test Generationmentioning

confidence: 99%

See 1 more Smart Citation

Mutation Testing Advances: An Analysis and Survey

Papadakis

Kintis

Zhang

et al. 2019

Advances in Computers

347

343

View full text Add to dashboard Cite

Mutation testing realises the idea of using artificial defects to support testing activities. Mutation is typically used as a way to evaluate the adequacy of test suites, to guide the generation of test cases and to support experimentation.Mutation has reached a maturity phase and gradually gains popularity both in academia and in industry. This chapter presents a survey of recent advances, over the past decade, related to the fundamental problems of mutation testing and sets out the challenges and open problems for the future development of the method. It also collects advices on best practices related to the use of mutation in empirical studies of software testing. Thus, giving the reader a 'mini-handbook'-style roadmap for the application of mutation testing as experimental methodology.

show abstract

Section: Identifying Equivalent Mutantsmentioning

confidence: 99%

Section: Static Constraint-based Test Generationmentioning

confidence: 99%

Mutation Testing Advances: An Analysis and Survey

Papadakis

Kintis

Zhang

et al. 2019

Advances in Computers

347

343

View full text Add to dashboard Cite

show abstract

“…reliance on dedicated hardware, emulators, and simulators also prevents the use of static program analysis to detect equivalent mutants [21], [22], [23]. Such characteristics are common to embedded software in other types of CPS domains including avionics, automotive, and industry 4.0 (e.g., robotics systems).…”

Section: Textmentioning

confidence: 99%

Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain

Olivares

Pastore

Briand

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

On-board embedded software developed for spaceflight systems (space software) must adhere to stringent software quality assurance procedures. For example, verification and validation activities are typically performed and assessed by third party organizations. To further minimize the risk of human mistakes, space agencies, such as the European Space Agency (ESA), are looking for automated solutions for the assessment of software testing activities, which play a crucial role in this context. Though space software is our focus here, it should be noted that such software shares the above considerations, to a large extent, with embedded software in many other types of cyber-physical systems. Over the years, mutation analysis has shown to be a promising solution for the automated assessment of test suites; it consists of measuring the quality of a test suite in terms of the percentage of injected faults leading to a test failure. A number of optimization techniques, addressing scalability and accuracy problems, have been proposed to facilitate the industrial adoption of mutation analysis. However, to date, two major problems prevent space agencies from enforcing mutation analysis in space software development. First, there is uncertainty regarding the feasibility of applying mutation analysis optimization techniques in their context. Second, most of the existing techniques either can break the real-time requirements common in embedded software or cannot be applied when the software is tested in Software Validation Facilities, including CPU emulators and sensor simulators. In this paper, we enhance mutation analysis optimization techniques to enable their applicability to embedded software and propose a pipeline that successfully integrates them to address scalability and accuracy issues in this context, as described above. Further, we report on the largest study involving embedded software systems in the mutation analysis literature. Our research is part of a research project funded by ESA ESTEC involving private companies (GomSpace Luxembourg and LuxSpace) in the space sector. These industry partners provided the case studies reported in this paper; they include an on-board software system managing a microsatellite currently on-orbit, a set of libraries used in deployed cubesats, and a mathematical library certified by ESA.

show abstract

“…Bardin et al proposed program verification to exclude test cases that cannot reach the mutant and/or that cannot infect the program state [4]. Other authors have explored using (static) symbolic execution techniques to identify whether a test case can detect mutants [15,28]. An example of a tool implementing this approach is PIT [8], that executes only those test cases that have a chance to kill the mutant, i.e.…”

Section: Existing Techniquesmentioning

confidence: 99%

“…Second, program verification excludes test cases which cannot reach the mutant and/or which cannot infect the program state [4]. Third, (static) symbolic execution techniques identify whether a test case is capable of killing the mutant [15,28]. This paper explores an alternative technique: fine-grained traceability links via focal methods [12].…”

Section: Introductionmentioning

confidence: 99%

Goal-oriented mutation testing with focal methods

Vercammen

Ghafari

Demeyer

et al. 2018

Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

View full text Add to dashboard Cite

Mutation testing is the state-of-the-art technique for assessing the fault-detection capacity of a test suite. Unfortunately, mutation testing consumes enormous computing resources because it runs the whole test suite for each and every injected mutant. In this paper we explore fine-grained traceability links at method level (named focal methods), to reduce the execution time of mutation testing and to verify the quality of the test cases for each individual method, instead of the usually verified overall test suite quality. Validation of our approach on the open source Apache Ant project shows a speed-up of 573.5x for the mutants located in focal methods with a quality score of 80

show abstract

Nequivack: Assessing Mutation Score Confidence

Cited by 12 publications

References 21 publications

Mutation Testing Advances: An Analysis and Survey

Mutation Testing Advances: An Analysis and Survey

Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain

Goal-oriented mutation testing with focal methods

Contact Info

Product

Resources

About