Non-parametric statistical fault localization

Abstract

Fault localization is a major activity in program debugging. To automate this time-consuming task, many existing fault-localization techniques compare passed executions and failed executions, and suggest suspicious program elements, such as predicates or statements, to facilitate the identification of faults. To do that, these techniques propose statistical models and use hypothesis testing methods to test the similarity or dissimilarity of proposed program features between passed and failed executions. Furthermore, when applying their models, these techniques presume that the feature spectra come from populations with specific distributions. The accuracy of using a model to describe feature spectra is related to and may be affected by the underlying distribution of the feature spectra, and the use of a (sound) model on inapplicable circumstances to describe real-life feature spectra may lower the effectiveness of these fault-localization techniques. In this paper, we make use of hypothesis testing methods as the core concept in developing a predicate-based fault-localization framework. We report a controlled experiment to compare, within our framework, the efficacy, scalability, and efficiency of applying three categories of hypothesis testing methods, namely, standard non-parametric hypothesis testing methods, standard parametric hypothesis testing methods, and debugging-specific parametric testing methods. We also conduct a case study to compare the effectiveness of the winner of these three categories with the effectiveness of 33 existing statement-level fault-localization techniques. The experimental results show that the use of non-parametric hypothesis testing methods in our proposed predicatebased fault-localization model is the most promising.

A general noise-reduction framework for fault localization of Java programs

“…Using code examining effort as a metric in the experiment may cause threats to the construct validity of the results. This has also been reported in previous projects [41][44] [45]. However, we are not aware of other popular metrics for evaluating the fault localization effectiveness.…”

“…Artzi et al [4] proposed an approach to automatically generating tests that expose failures, which also facilitate fault localization by alleviating the limitation of previous fault-localization techniques that a test suite must be available upfront. Zhang et al [45] proposed nonparametric predicate-based statistical fault-localization framework, which also study its effectiveness on statement-level base techniques listed in [28]. Moreover, it also report experiments on predicate-based techniques and some other fault-localization techniques.…”

“…Existing work has proposed many similarity coefficients [1][4] [24][25] [30][39] [41] or derived coefficients [31] [45]. Many experiments have been conducted on these different coefficients under various benchmarks to compare their effectiveness.…”

A Clustering-Based Strategy to Identify Coincidental Correctness in Fault Localization

“…(2) Figure 5 shows the e®ectiveness improvement (ÁTS) of the remained 18 versions. The effectiveness improvement on some versions (13,17,24,26) are signi¯cant. All results are positive except the cleansing strategy on version 29.…”

A theoretical analysis on cloning the failed test cases to improve spectrum-based fault localization

“…In particular, there is a large body of research on fault localization. Spectrum-Based Fault Localization (SBFL) is one of them under active study in the recent decades (Jones et al, 2002;Liblit et al, 2005;Zhang et al, 2005;Abreu et al, 2006Abreu et al, , 2007Wong et al, 2007Wong et al, , 2008Zhang et al, 2009;Naish et al, 2011;Zhang et al, 2011;Wong et al, 2012;Xie et al, 2013a,b;Xu et al, 2013;Wong et al, 2014).…”