Bug numbers matter: An empirical study of effort‐aware defect prediction using class labels versus bug numbers

Abstract

Previous research have utilized public software defect datasets such as NASA, RELINK, and SOFTLAB, which only contain class label information. Most effort‐aware defect prediction (EADP) studies are carried out around these datasets. However, EADP studies typically relying on predicted bug number (i.e., considering modules as effort) or density (i.e., considering lines of code as effort) for ranking software modules. To explore the impact of bug number information in constructing EADP models, we access the performance degradation of the best‐performing learning‐to‐rank methods when using class labels instead of bug numbers for training. The experimental results show that using class labels instead of bug numbers in building EADP models results in an decrease in the detected bugs when module is considering as effort. When effort is LOC, using class labels to construct EADP models can lead to a significant increase in the initial false alarms and a significant increase in the modules that need to be inspected. Therefore, we recommend not only the class labels but also the bug number information should be disclosed when publishing software defect datasets, in order to construct more accurate EADP models.