Where is the bug and how is it fixed? an experiment with practitioners

Böhme, Marcel; Soremekun, Ezekiel; Chattopadhyay, Sudipta; Ugherughe, Emamurho; Zeller, Andreas

doi:10.1145/3106237.3106255

Cited by 88 publications

(65 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SBFL techniques often assign the same suspiciousness scores for multiple lines or code elements Users of SBFL techniques do not inspect in the order provided by the ranked list SBFL techniques require a large number of failing test cases to accurately reveal a fault …”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

et al. 2019

View full text Add to dashboard Cite

Pure spectrum-based fault localization (SBFL) is a well-studied statistical debugging technique that only takes a set of test cases (some failing and some passing) and their code coverage as input and produces a ranked list of suspicious program elements to help the developer identify the location of a bug that causes a failed test case. Studies show that pure SBFL techniques produce good ranked lists for small programs. However, our previous study based on the iBugs benchmark that uses the AspectJ repository shows that, for realistic programs, the accuracy of the ranked list is not suitable for human developers. In this paper, we confirm this based on a combined empirical evaluation with the iBugs and the Defects4J benchmark. Our experiments show that, on average, at most ∼40%, ∼80%, and ∼90% of the bugs can be localized reliably within the first 10, 100, and 1000 ranked lines, respectively, in the Defects4J benchmark.To reliably localize 90% of the bugs with the best performing SBFL metric D * , ∼450 lines have to be inspected by the developer. For human developers, this remains unsuitable, although the results improve compared with the results for the AspectJ benchmark. Based on this study, we can clearly see the need to go beyond pure SBFL and take other information, such as information from the bug report or from version history of the code lines, into consideration.

show abstract

Section: Methodsmentioning

confidence: 99%

“…However, while tests are effective at exposing erroneous behavior, identifying the root cause behind a failure remains mostly a manual activity, requiring significant time and costs …”

Section: Introductionmentioning

confidence: 99%

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

et al. 2019

View full text Add to dashboard Cite

show abstract

“…During the sessions, the author observed their manual debugging practice and categorized it under the "Bug Diagnostic Strategies" explained in [4]. In addition, the number of executions performed by the interviewee, manually or by the tool, was recorded.…”

Section: Interview Design and Setupmentioning

confidence: 99%

“…Q 3.1.3: Which diagnostic strategies were used? Answer: The three developers used a mix of three debugging strategies: Forward Reasoning, Backward Reasoning, and Code Comprehension, as defined in [4]. In forward reasoning, the developer starts from the starting point of the execution and move towards the first occurrence of the issue in search of the cause of the issue, while executing the source code.…”

Section: Q135: Assume You Select the Best Set Of Fields And Functiomentioning

confidence: 99%

Interactive semi-automated specification mining for debugging: An experience report

Mashhadi

Siddiqui

Hemmati

et al. 2019

Information and Software Technology

View full text Add to dashboard Cite

Context: Specification mining techniques are typically used to extract the specification of a software in the absence of (up-to-date) specification documents. This is useful for program comprehension, testing, and anomaly detection. However, specification mining can also potentially be used for debugging, where a faulty behavior is abstracted to give developers a context about the bug and help them locating it. Objective: In this project, we investigate this idea in an industrial setting. We propose a very basic semi-automated specification mining approach for debugging and apply that on real reported issues from an AutoPilot software system from our industry partner, MicroPilot Inc. The objective is to assess the feasibility and usefulness of the approach in a real-world setting. Method: The approach is developed as a prototype tool, working on C code, which accept a set of relevant state fields and functions, per issue, and generates an extended finite state machine that represents the faulty behavior, abstracted with respect to the relevant context (the selected fields and functions). Results:We qualitatively evaluate the approach by a set of interviews (including observational studies) with the company's developers on their real-world reported bugs. The results show that a) our approach is feasible, b) it can be automated to some extent, and c) brings advantages over only using their code-level debugging tools. We also compared this approach with traditional fully automated state-merging algorithms and reported several issues when applying those techniques on a real-world debugging context. Conclusion: The main conclusion of this study is that the idea of an "interactive" specification mining rather than a fully automated mining tool is NOT impractical and indeed is useful for the debugging use case.

show abstract

“…Several methods in the literature have been used to distinguish debugging from software development. In many studies, people look at version control history messages and comments to find bug fixing commits and edits [2]. In others, a bug is indicated by failing test cases in the project [6].…”

Section: Data Processingmentioning

confidence: 99%

A Study on the Use of IDE Features for Debugging

Afzal¹,

Goues²

2018

EasyChair Preprints

View full text Add to dashboard Cite

Integrated development environments (IDEs) provide features to help developers both create and understand code. As maintenance and bug repair are time-consuming and costly activities, IDEs have long integrated debugging features to simplify these tasks. In this paper we investigate the impact of using IDE debugger features on different aspects of programming and debugging. Using the data set provided by MSR challenge track, we compared debugging tasks performed with or without the IDE debugger. We find, on average, that developers spend more time and effort on debugging when they use the debugger. Typically, developers start using the debugger early, at the beginning of a debugging session, and that their editing behavior does not appear to significantly change when they are debugging regardless of whether debugging features are in use.

show abstract

Where is the bug and how is it fixed? an experiment with practitioners

Cited by 88 publications

References 44 publications

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

Interactive semi-automated specification mining for debugging: An experience report

A Study on the Use of IDE Features for Debugging

Contact Info

Product

Resources

About