Statistical Fault Localisation (SFL) is a widely used method for localizing faults in software. SFL gathers coverage details of passed and failed executions over a faulty program and then uses a measure to assign a degree of suspiciousness to each of a chosen set of program entities (statements, predicates, etc.) in that program. The program entities are then inspected by the engineer in descending order of suspiciousness until the bug is found. The effectiveness of this process relies on the quality of the suspiciousness measure. In this paper, we compare 157 measures, 95 of which are new to SFL and borrowed from other branches of science and philosophy. We also present a new measure optimiser Lex g , which optimises a given measure g according to a criterion of single bug optimality. An experimental comparison on benchmarks from the Software-artifact Infrastructure Repository (SIR) indicates that many of the new measures perform competitively with the established ones. Furthermore, the large-scale comparison reveals that the new measures Lex Ochiai and Pattern-Similarity perform best overall.

Abstract. Efficient fault localisation is becoming increasingly important as software grows in size and complexity. In this paper we present a new formal framework, denoted probabilistic fault localisation (pfl), and compare it to the established framework of spectrum based fault localisation (sbfl). We formally prove that pfl satisfies some desirable properties which sbfl does not, empirically demonstrate that pfl is significantly more effective at finding faults than all known sbfl measures in large scale experimentation, and show pfl has comparable efficiency. Results show that the user investigates 37% more code (and finds a fault immediately in 27% fewer cases) when using the best performing sbfl measures, compared to the pfl framework. Furthermore, we show that it is theoretically impossible to design strictly rational sbfl measures that outperform pfl techniques on a large set of benchmarks.

Spectrum based fault localisation determines how suspicious a line of code is with respect to being faulty as a function of a given test suite. Outstanding problems include identifying properties that the test suite should satisfy in order to improve fault localisation effectiveness subject to a given measure, and developing methods that generate these test suites efficiently.We address these problems as follows. First, when single bug optimal measures are being used with a single-fault program, we identify a formal property that the test suite should satisfy in order to optimise fault localisation. Second, we introduce a new method which generates test data that satisfies this property. Finally, we empirically demonstrate the utility of our implementation at fault localisation on sv-comp benchmarks and the tcas program, demonstrating that test suites can be generated in almost a second with a fault identified after inspecting under 1% of the program.