Eliminating the high false‐positive rate in defect prediction through BayesNet with adjustable weight

Abstract

In defect prediction, a high false‐positive rate (FPR) caused by class imbalance not only increases the workload of testing and development but also consumes unnecessary costs. Many defect models against class imbalance have been proposed to improve the accuracy of defect prediction, but their ability to reduce FPR is unclear. To solve these problems, we first proposed a BayesNet with adjustable weights, called WBN, to reduce the FPR in software defect prediction, which is an algorithm independent of data preprocessing techniques. The mechanism of our WBN is to change the sampling probability of the misclassified instances when training the defect model, making the BayesNet model focus more on false alarm instances. And then, we investigate the FPR of five mainstream defect models for solving class imbalance and select them as comparison models to test the validity of our methods. The experimental result on eight open‐source projects shows that a) our WBN, in in‐version defect prediction (IVDP) and cross‐version defect prediction (CVDP), effectively reduces FPR with means of 0.384 and 0.322, respectively; b) compared with improved subclass discriminant analysis (ISDA) that is the lowest FPR in all control models, our WBN not only reduced the FPR but maintained recall whose mean value was 0.797, whereas ISDA did not, with an average recall of only 0.397; c) our WBN, in CVDP, not only reduces FPR, but also has significant superiority over five control defect models and baseline. Besides, we also found that the class imbalance difference between the test set and the training set has an impact on CVDP performance, recommending that practitioners choose the best dataset for CVDP from the defect data of the historical version through special technology.

Evaluating the Impact of Data Transformation Techniques on the Performance and Interpretability of Software Defect Prediction Models

“…Similar to classification techniques, these indicators also take into account the requirements of quantity and type, and these indicators have been widely used in the previous defect prediction research [86,87]. Nevertheless, more other types of indicators are not used in this paper [88][89][90], such as Gmeasure and FPR, etc. Recently effort-aware indicators have also been proposed to measure the performance of SDP models [91][92][93].…”

Bug Prediction Models: seeking the most efficient

“…The higher rate of false positives that the CART can generate has a detrimental effect on the amount of time and resources required to investigate false positives, which are non-buggy classes mistakenly labeled as buggy (Yanyang et al, 2022). Its poor F1score of 0.9114, which is exceeded by the RF (0.9169), provides additional evidence that this is the case.…”

A Novel Supervised Filter Feature Selection Method Based on Gaussian Probability Density for Fault Diagnosis of Permanent Magnet DC Motors

“…The samples of each category are randomly divided into 70% and 30%, samples are used to search the optimal hyperparameters of the SVM classifier, and the remaining samples are used to test the fault diagnosis accuracy of the PMDCMs. For each fault diagnosis, the accuracy of PMDCM’s model is checked by utilizing the combination of and ; the 10-fold cross-validation [ 43 ] is utilized while the aim is to obtain stable fault diagnosis accuracy. The process of 10-fold cross-validation can be seen in Figure 13 .…”