Works for me! characterizing non-reproducible bug reports

Abstract

Bug repository systems have become an integral component of software development activities. Ideally, each bug report should help developers to find and fix a software fault. However, there is a subset of reported bugs that is not (easily) reproducible, on which developers spend considerable amounts of time and e↵ort. We present an empirical analysis of nonreproducible bug reports to characterize their rate, nature, and root causes. We mine one industrial and five opensource bug repositories, resulting in 32K non-reproducible bug reports. We (1) compare properties of non-reproducible reports with their counterparts such as active time and number of authors, (2) investigate their life-cycle patterns, and (3) examine 120 Fixed non-reproducible reports. In addition, we qualitatively classify a set of randomly selected non-reproducible bug reports (1,643) into six common categories. Our results show that, on average, non-reproducible bug reports pertain to 17% of all bug reports, remain active three months longer than their counterparts, can be mainly (45%) classified as "Interbug Dependencies", and 66% of Fixed non-reproducible reports were indeed reproduced and fixed.

Auto-completing bug reports for Android applications

“…A significant amount of research has been conducted concerning the summarization [51,23,63,47,73,32], fault localization [77,71,62,19,70,75,52,17,31,33], classification and detection of duplicate bug reports [35,59,72,39,76,38,61]. Again, the work presented in this paper compliments these categories of research as bug reports created with FUSION can provide more detailed information, easily linking the bug back to source code, allowing for better localization, summarization and, potentially, duplicate detection.…”

“…This highlights the underlying task that bug reporting systems must accomplish: bridging the lexical knowledge gap between typical reporters of a bug and the developers that must resolve the bugs. Previous studies on bug report quality and developer information needs highlight several factors that can impact the quality of bug reports [25,35,21]:…”

“…• Other than "Interbug dependencies" (i.e., a situation where a bug was fixed in a previous patch), insufficient information is one of the leading causes of nonreproducible bug reports [35];…”

“…Several participants complained of overly specific or detailed information during the second study, and this information may have been more suited to a fault-location task. In our study we focused on reproduction to gauge bug report quality as it is well known that if a developer can reproduce a bug there is a much higher chance that they will be able to fix and patch it [35,21,25]. However, based on the user experience and preference results from Study 2, it may be beneficial to present information to developers in stages (e.g., first present reproduction steps, then more detailed code-information for fault location).…”

Identifying and predicting key features to support bug reporting

“…We note that Steps to Reproduce and Test Case are requested most often. In reality, Steps to Reproduce is useful because it enables developers to reproduce and understand the bugs. Sometimes, the developers cannot fix a bug without Steps to Reproduce .…”