Information Retrieval Based Nearest Neighbor Classification for Fine-Grained Bug Severity Prediction

Abstract

Bugs are prevalent in software systems. Some bugs are critical and need to be fixed right away, whereas others are minor and their fixes could be postponed until resources are available. In this work, we propose a new approach leveraging information retrieval, in particular BM25-based document similarity function, to automatically predict the severity of bug reports. Our approach automatically analyzes bug reports reported in the past along with their assigned severity labels, and recommends severity labels to newly reported bug reports. Duplicate bug reports are utilized to determine what bug report features, be it textual, ordinal, or categorical, are important. We focus on predicting fine-grained severity labels, namely the different severity labels of Bugzilla including: blocker, critical, major, minor, and trivial. Compared to the existing state-of-the-art study on fine-grained severity prediction, namely the work by Menzies and Marcus, our approach brings significant improvement.

DRONE: Predicting Priority of Reported Bugs by Multi-factor Analysis

“…Closest to our work, is the series of work on bug report severity prediction by Menzies and Marcus [16], Lamkanfi et al [13], [14], and our own previous work [27]. These studies predict the severity field of a bug report based on the textual content of the report.…”

“…Recently, Tian et al also predict the severity of bug reports by utilizing a nearest neighbor approach to predict fine grained bug report labels [27]. Different from the work by Menzies and Marcus which analyzes a collection of bug reports in NASA, Tian et al apply the solution on a larger collection of bug reports consisting of more than 65,000 Bugzilla reports.…”

An Empirical Study of Bug Fixing Rate

“…Tian et at. [25] use information retrieval based nearest neighbor classification to conduct the bug severity prediction.…”

An Empirical Study of Bugs in Software Build System

“…It is notable that in JIRA (Maven, QMake), the default severity level when a user creates a new bug report is major, while in Bugzilla (Ant), the default severity level is normal, and in MantisBT (CMake), the default severity level is minor. Past studies have shown that the number of bug reports with default severity level is the largest [19], [22]- [24]. Herraiz et al state that the many severity levels can confuse bug reporters which are often unable to distinguish the meaning of the different severity levels [25].…”

“…et al argue that bug reporters who do not assess bug severity level well often assign these bugs to the default severity level [22]. As a future work, we could explore automated methods (e.g., [19], [22]- [24]) to recommend more accurate severity levels to bugs before performing a more detailed analysis. Following the definition of the various severity levels, block bug refers to a bug that causes system crash, data corruption, irreparable harm, etc., and critical bug refers to a bug that affects an important function and it has no reasonable workaround.…”