Leveraging developer information for efficient effort-aware bug prediction

Qu, Yu; Chi, Jianlei; Yin, Heng

doi:10.1016/j.infsof.2021.106605

Cited by 16 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research has shown that defects can be influenced by the characteristics and experience levels of developers (Qu et al, 2021). Considering this metric, we aim to explore the relationship between team dynamics and defect proneness.…”

Section: Data Collectionmentioning

confidence: 99%

Bug Prediction Models: seeking the most efficient

Marçal,

Garcia

2024

Preprint

View full text Add to dashboard Cite

Choosing the most appropriate machine learning model for bug prediction tasks is critical. This paper primarily compares the predictive power of individual models versus ensemble models. We begin by experimenting with popular single-machine learning models commonly used in bug prediction, like neural networks and support vector machines. Additionally, we test with ensemble models that combine individual models' unique strengths, aiming to maximize each's benefits. Our evaluation is based on datasets containing historical development data from well-known open-source software projects. We rely on various metrics when assessing the models, encompassing accuracy, precision, recall, and F1 score. Based on our research findings, it has been observed that ensemble models tend to outperform single models, particularly when it comes to maintaining resilience across various datasets.Nevertheless, factors like the project's complexity, data availability, and computational resources all play a role in determining whether to use single or ensemble models. This paper offers a thorough analysis of the factors to consider when selecting machine learning models and approaches for bug prediction, providing valuable insights into the field. Furthermore, it offers practical advice for professionals, enabling them to make informed choices.

show abstract

Section: Data Collectionmentioning

confidence: 99%

Bug Prediction Models: seeking the most efficient

Marçal,

Garcia

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Qu et al [61] proposed a top-core equation to help rearrange the likely defective modules for EADP. Qu et al [62] proposed integrating developer information into EADP to enhance performance. Carka et al [63] proposed to assess the EADP performance using the normalised PofB, which sorted software modules according to the predicted defect densities.…”

Section: Effort-aware Defect Predictionmentioning

confidence: 99%

“…Qu et al. [62] proposed integrating developer information into EADP to enhance performance. Carka et al.…”

Section: Related Workmentioning

confidence: 99%

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

Keung

et al. 2023

IET Software

View full text Add to dashboard Cite

Effort‐Aware Defect Prediction (EADP) methods sort software modules based on the defect density and guide the testing team to inspect the modules with high defect density first. Previous studies indicated that some feature selection methods could improve the performance of Classification‐Based Defect Prediction (CBDP) models, and the Correlation‐based feature subset selection method with the Best First strategy (CorBF) performed the best. However, the practical benefits of feature selection methods on EADP performance are still unknown, and blindly employing the best‐performing CorBF method in CBDP to pre‐process the defect datasets may not improve the performance of EADP models but possibly result in performance degradation. To assess the impact of the feature selection techniques on EADP, a total of 24 feature selection methods with 10 classifiers embedded in a state‐of‐the‐art EADP model (CBS+) on the 41 PROMISE defect datasets were examined. We employ six evaluation metrics to assess the performance of EADP models comprehensively. The results show that (1) The impact of the feature selection methods varies in classifiers and datasets. (2) The four wrapper‐based feature subset selection methods with forwards search, that is, AdaBoost with Forwards Search, Deep Forest with Forwards Search, Random Forest with Forwards Search, and XGBoost with Forwards Search (XGBF) are better than other methods across the studied classifiers and the used datasets. And XGBF with XGBoost as the embedded classifier in CBS+ performs the best on the datasets. (3) The best‐performing CorBF method in CBDP does not perform well on the EADP task. (4) The selected features vary with different feature selection methods and different datasets, and the features noc (number of children), ic (inheritance coupling), cbo (coupling between object classes), and cbm (coupling between methods) are frequently selected by the four wrapper‐based feature subset selection methods with forwards search. (5) Using AdaBoost, deep forest, random forest, and XGBoost as the base classifiers embedded in CBS+ can achieve the best performance. In summary, we recommend the software testing team should employ XGBF with XGBoost as the embedded classifier in CBS+ to enhance the EADP performance.

show abstract

“…Specifically, a larger PMI@20% indicates that developers need to inspect more files under the same volume of LOC to inspect. Thus bug prediction models should strive to reduce PMI@20% while trying to increase Popt (Qu et al 2021b) at the same time. -PFI@20%: has been introduced by Qu et al (2021b) and it coincides with PMI@20 when the module is a file.…”

Section: Effort-aware Metricsmentioning

confidence: 99%

“…Thus bug prediction models should strive to reduce PMI@20% while trying to increase Popt (Qu et al 2021b) at the same time. -PFI@20%: has been introduced by Qu et al (2021b) and it coincides with PMI@20 when the module is a file.…”

Section: Effort-aware Metricsmentioning

confidence: 99%

On effort-aware metrics for defect prediction

2022

View full text Add to dashboard Cite

Context Advances in defect prediction models, aka classifiers, have been validated via accuracy metrics. Effort-aware metrics (EAMs) relate to benefits provided by a classifier in accurately ranking defective entities such as classes or methods. PofB is an EAM that relates to a user that follows a ranking of the probability that an entity is defective, provided by the classifier. Despite the importance of EAMs, there is no study investigating EAMs trends and validity. Aim The aim of this paper is twofold: 1) we reveal issues in EAMs usage, and 2) we propose and evaluate a normalization of PofBs (aka NPofBs), which is based on ranking defective entities by predicted defect density. Method We perform a systematic mapping study featuring 152 primary studies in major journals and an empirical study featuring 10 EAMs, 10 classifiers, two industrial, and 12 open-source projects. Results Our systematic mapping study reveals that most studies using EAMs use only a single EAM (e.g., PofB20) and that some studies mismatched EAMs names. The main result of our empirical study is that NPofBs are statistically and by orders of magnitude higher than PofBs. Conclusions In conclusion, the proposed normalization of PofBs: (i) increases the realism of results as it relates to a better use of classifiers, and (ii) promotes the practical adoption of prediction models in industry as it shows higher benefits. Finally, we provide a tool to compute EAMs to support researchers in avoiding past issues in using EAMs.

show abstract

Leveraging developer information for efficient effort-aware bug prediction

Cited by 16 publications

References 36 publications

Bug Prediction Models: seeking the most efficient

Bug Prediction Models: seeking the most efficient

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

On effort-aware metrics for defect prediction

Contact Info

Product

Resources

About