Bug or Not? Bug Report Classification Using N-Gram IDF

Abstract

Previous studies have found that a significant number of bug reports are misclassified between bugs and nonbugs, and that manually classifying bug reports is a timeconsuming task. To address this problem, we propose a bug reports classification model with N-gram IDF, a theoretical extension of Inverse Document Frequency (IDF) for handling words and phrases of any length. N-gram IDF enables us to extract key terms of any length from texts, these key terms can be used as the features to classify bug reports. We build classification models with logistic regression and random forest using features from N-gram IDF and topic modeling, which is widely used in various software engineering tasks. With a publicly available dataset, our results show that our N-gram IDF-based models have a superior performance than the topic-based models on all of the evaluated cases. Our models show promising results and have a potential to be extended to other software engineering tasks.

On the feasibility of automated prediction of bug and non-bug issues

“…Zolkeply and Shao (2019) propose to not use TFM frequencies, but simply the occurrence of one of 60 keywords as binary features and use these to train a Classification Association Rule Mining (CARM). Terdchanakul et al (2017) propose to go beyond the TFM and instead use the Inverse Document Frequency (IDF) of n-grams for the title and descriptions of the issues as input for either LR or RF as classifier. Zhou et al (2016) propose an approach that combines the TFM from the issue title with structured information about the issue, e.g., the priority and the severity.…”

“…We implemented the approaches as they were described and refer to them by the family name of the first author, year of publication, and acronym for the classifier. The approaches from the literature we consider are (in alphabetatical order) Kallis2019-FT by Kallis Limsettho et al (2014a), and Terdchanakul2017-LR and Terdchanakul2017-RF by Terdchanakul et al (2017).…”

“…Researchers suggested an alternative through the automated classification of issue types by analyzing the issue titles and descriptions with unsupervised machine learning based on clustering the issues Limsettho et al (2014b), Hammad et al (2018) and Chawla and Singh (2018) and supervised machine learning that create classification models (Antoniol et al 2008;Pingclasai et al 2013;Limsettho et al 2014a;Chawla and Singh 2015;Zhou et al 2016;Terdchanakul et al 2017;Pandey et al 2018;Qin and Sun 2018;Zolkeply and Shao 2019;Otoom et al 2019;Kallis et al 2019). There are two possible use cases for such automated classification models.…”

Improving bug report triage performance using artificial intelligence based document generation model

“…To resolve these problems, artificial intelligence techniques are now actively being studied, and have shown better classification accuracy than traditional (non-artificial intelligence based) methods [25][26][27][28][29][30][31][32][33][34][35][36]. Such techniques can be a key to solving most of the current problems regarding this issue.…”

“…To reduce the effort required in this regard, studies have proposed the application of state-of-the-art automation methods for bug report classification [25][26][27][28][29]. In particular, latent Dirichlet allocation (LDA)-based classification methods are common because they are suitable to bug reports that contain text-based data.…”