Simultaneous debugging of software faults

Abstract

Semi-)automated diagnosis of software faults can drastically increase debugging efficiency, improving reliability and time-to-market. Current automatic diagnosis techniques are predominantly of a statistical nature and, despite typical defect densities, do not explicitly consider multiple faults, as also demonstrated by the popularity of the single-fault benchmark set of programs. We present a reasoning approach, called Zoltar-M(ultiple fault), that yields multiple-fault diagnoses, ranked in order of their probability. Although application of Zoltar-M to programs with many faults requires heuristics (trading-off completeness) to reduce the inherent computational complexity, theory as well as experiments on synthetic program models and multiple-fault program versions available from the software infrastructure repository (SIR) show that for multiple-fault programs this approach can outperform statistical techniques, notably spectrum-based fault localization (SFL). As a side-effect of this research, we present a new SFL variant, called Zoltar-S(ingle fault), that is optimal for single-fault programs, outperforming all other variants known to date.

Probabilistic reasoning in diagnosing causes of program failures

“…In addition, the two effort metrics mentioned in Section 5.1.2 are taken to compare PCEG with five earlier techniques that are often used for evaluating fault localization techniques. Among the five techniques, Tarantula and Ochiai feature statement‐level coverage information, LOUPE features run‐time dependency coverage information, PPDG is based on a probabilistic graphical model, and BARINEL combines SFL with a probabilistic reasoning approach. RQ2. How does the accuracy of PCEG compare to available techniques with real‐larger programs?If it has been answered by RQ1 that PCEG is effective on Siemens benchmark, it would be wished that it performed on the larger and realistic programs, such as space and grep .…”

“…RQ3 concerns with the time complexity of PCEG when it compares with that of other techniques. For one thing, as most fault localization techniques did in the context of single fault, PCEG is compared with others in single‐fault case and obtains the performance data of PPDG, BARINEL, Tarantula and Ochiai from the literature . For another thing, because PCEG has been compared with re‐implemented LOUPE, Ochiai, Tarantula and BARINEL, a direct comparison of PCEG, LOUPE, Ochiai, Tarantula and BARINEL is given in terms of time‐cost on the same computer equipment for the subjects shown in Table .…”

“…Most recently, probabilistic graph model and probabilistic reasoning techniques have been adapted into software fault diagnosis and attracted more and more attention. Abreu and Wotawa proposed an extension of consistency‐based models with probabilistic reasoning technique to localizing faults. Baah presented a model called probabilistic program dependency graph (PPDG) to estimate the states of the variables and capture the conditional dependencies and independencies in passed and failed runs to facilitate fault localization by probabilistic reasoning .…”

“…The probabilistic reasoning of PCEG features a bidirectional inference; while top‐down inference calculates the probability distribution of apparent fault nodes, bottom‐up inference updates the posterior probability distribution of real fault nodes. The PCEG approach utilizes run‐time dependencies between possible faulty statements and can decide whether a fault occurred in control flow or in data flow, so it is especially helpful in understanding the types of fault and its root causes. To validate PCEG approach, extensive experiments have been performed on the well‐known Siemens benchmark. By comparison, PCEG approach is more effective in fault localization than Tarantula , Ochiai , PPDG , BARINEL and LOUPE . The empirical results have shown the improvements in terms of diagnostic effectiveness and accuracy.…”

Effective Fault Localization Approach Using Feedback

“…To support locating multiple faults in parallel, Jones et al [15] adopt a clustering technique to divide failed test cases into different clusters and each cluster represents one fault. Abreu et al [16] propose a spectrum-based multiple fault localization approach called Zoltar-M that integrates SFL with model-based diagnosis. To address the coincidental correctness problem in coverage-based fault localization approaches, Wang et al [17] prescribe patterns for different types of faults and modify the coverage vectors to locate faults with the feature of coincidental correctness.…”