Efficient mutation testing of multithreaded code

Abstract

SUMMARYMutation testing is a well‐established method for measuring and improving the quality of test suites. A major cost of mutation testing is the time required to execute the test suite on all the mutants. This cost is even greater when the system under test is multithreaded: not only are test cases from the test suite executed on many mutants but also each test case is executed—or more precisely, explored—for multiple possible thread schedules. This paper introduces a general framework for efficient exploration that can reduce the time for mutation testing of multithreaded code. The paper presents five techniques (four optimizations and one heuristic) that are implemented in a tool called MuTMuT within the general framework. Evaluation of MuTMuT on mutation testing of 12 multithreaded programs shows that it can substantially reduce the time required for mutation testing of multithreaded code.Copyright © 2012 John Wiley & Sons, Ltd.

Mutation Testing Advances: An Analysis and Survey

“…For instance, Gligoric et al [207,208] suggest a method for the efficient state-space exploration of multithreaded programs. This work involves optimisation techniques and heuristics that achieve substantial mutant execution savings.…”

Um framework para avaliação sistemática de técnicas de teste no contexto de programação concorrente

“…• Results show that sequential and concurrent mutation operators are independent which demonstrates the importance of studying concurrent mutation operators [16]. An orthogonal optimization to selective mutation called MuT-MuT [59,29] for mutation testing of concurrent code optimizes the execution of mutants up to 77% by reducing the state space explored.…”

“…Abordagens são propostas também para a técnica baseada em defeitos, como o teste de mutação (GLIGORIC et al, 2010;SILVA et al, 2012;GLIGORIC et al, 2013).…”

“…Alowibdi and Stenneth [28] An empirical study of data race detector tools Souza et al [14] Empirical evaluation of a new composite approach to the coverage criteria and reachability testing of concurrent programs Gligoric et al [29] E cient mutation testing of multithreaded code Lu et al [30] Learning from mistakes: a comprehensive study on real world concurrency bug characteristics Koppol et al [31] Incremental integration testing of concurrent programs of technique and the main conclusions drawn from the studies. These subsections also provide a list of the primary limitations of the set of studies.…”

“…A commonly used fault-based technique is the mutation testing which uses knowledge about common mistakes made by developers to generate mutant programs, which are similar to the original but with minor syntactical changes produced by the mutant operators. Concurrent software requires additional mutation operators to handle communication and synchronization mistakes and to seek alternatives for deterministic re-execution [16,59,29,60]. The main conclusions reached by this group of studies are:…”