No abstract

International audienceSoftware developers employ many tools in every step of the development. As automation progresses, tools take a more and more important place. A common and difficult problem is choosing a tool among every tool for a given task.As a particular instance of this problem, this paper considers mutation analysis tools. Mutation analysis is a way to evaluate the quality of a test suite. The quality is measured as the ability of the test suite to detect faults injected into the program under tests. A fault is detected if at least one test case gives different results on the original program and the fault-injected one. Mutation tools aim at automating and speeding both the generation of fault-injected variants, called mutants, and the execution of the test suite on those mutants.This paper proposes a methodology to compare tools and applies it for comparing mutation tools. This methodology proposes to dress a list of comparison criteria as well as a list of usage profiles. Mutation tools for Java are compared on paper and by experiments. The work is then extended to other languages to assert the pertinence of the comparison criteria and the usage profiles. Finally, lessons are drawn from our selection process

Abstract. TOBIAS is a combinatorial testing tool, aimed at the production of large test suites. In this paper, TOBIAS is applied to conformance tests for model-based specifications (expressed with assertions, pre and post-conditions) and associated implementations. The tool takes advantage of the executable character of VDM or JML assertions which provide an oracle for the testing process. Executing large test suites may require a lot of time. This paper shows how assertions can be exploited at generation time to filter the set of test cases, and at execution time to detect inconclusive test cases.