Interactive Fault Localization Using Test Information

Abstract

Debugging is a time-consuming task in software development. Although various automated approaches have been proposed, they are not effective enough. On the other hand, in manual debugging, developers have difficulty in choosing breakpoints. To address these problems and help developers locate faults effectively, we propose an interactive fault-localization framework, combining the benefits of automated approaches and manual debugging. Before the fault is found, this framework continuously recommends checking points based on statements' suspicions, which are calculated according to the execution information of test cases and the feedback information from the developer at earlier checking points. Then we propose a naive approach, which is an initial implementation of this framework. However, with this naive approach or manual debugging, developers' wrong estimation of whether the faulty statement is executed before the checking point (breakpoint) may make the debugging process fail. So we propose another robust approach based on this framework, handling cases where developers make mistakes during the fault-localization process. We performed two experimental studies and the results show that the two interactive approaches are quite effective compared with existing fault-localization approaches. Moreover, the robust approach can help developers find faults when they make wrong estimation at some checking points.

Effective Fault Localization Approach Using Feedback

“…To evaluate the effectiveness of our approach, the experimental study chooses the Siemens suite and Space as our benchmarks because they are two widely used benchmarks in the field of fault localization [2], [4], [8], [9], [11], [12], [15]- [21], [25]. They cover a wide spectrum of faults with high quality, such as predicate faults, assignment faults, missing code, etc.…”

“…In addition, two factors that influence interactive performance are also analyzed in [24]. Hao et al [25] use breakpoints to interact with debugging engineers and can somewhat rectify mistakes made by engineers. The above interactive fault localization approaches still need intensive workload of debugging engineers in analysis and deduction which incurs a lot of overhead.…”

“…A typical process of debugging consists of four steps: failure reproduction, fault localization, fault repair and repair verification, among which fault localization is usually a tedious and difficult step [2]. With the aim at decreasing the cost of debugging, many researchers try to optimize the process of fault localization and improve its performance [1]- [25].…”

“…At present, some interactive fault localization approaches have been already presented that can interact with debugging engineers [22]- [25]. Because the interaction reflects the brainpower of debugging engineers, it is genuinely difficult for fault localization approaches to analyze and deduce like engineers.…”

Is This a Bug or an Obsolete Test?

“…Most existing research [2,22,52,64] on software debugging focuses on the first step, which is fault localization. Typically, spectrum-based fault-localization approaches [20,27,34] compare the execution information of failed tests and that of passed tests to calculate the suspiciousness of each structural units, and then localize the locations of faulty structural units by ranking the structural units based on their suspiciousness.…”