Revisiting unsupervised learning for defect prediction

Wei, Fu; Menzies, Tim

doi:10.1145/3106237.3106257

Cited by 109 publications

(80 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…eir study showed that CCUM performs better than the state-of-the-art prediction model at that time. As the results Yang et al [8] reported are startling, Fu et al [28] repeated their experiment.…”

Section: Effort-aware Software Defect Predictionmentioning

confidence: 91%

Local versus Global Models for Just-In-Time Software Defect Prediction

Yang

Fan

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

Just-in-time software defect prediction (JIT-SDP) is an active topic in software defect prediction, which aims to identify defect-inducing changes. Recently, some studies have found that the variability of defect data sets can affect the performance of defect predictors. By using local models, it can help improve the performance of prediction models. However, previous studies have focused on module-level defect prediction. Whether local models are still valid in the context of JIT-SDP is an important issue. To this end, we compare the performance of local and global models through a large-scale empirical study based on six open-source projects with 227417 changes. The experiment considers three evaluation scenarios of cross-validation, cross-project-validation, and timewise-cross-validation. To build local models, the experiment uses the k-medoids to divide the training set into several homogeneous regions. In addition, logistic regression and effort-aware linear regression (EALR) are used to build classification models and effort-aware prediction models, respectively. The empirical results show that local models perform worse than global models in the classification performance. However, local models have significantly better effort-aware prediction performance than global models in the cross-validation and cross-project-validation scenarios. Particularly, when the number of clusters k is set to 2, local models can obtain optimal effort-aware prediction performance. Therefore, local models are promising for effort-aware JIT-SDP.

show abstract

Section: Effort-aware Software Defect Predictionmentioning

confidence: 91%

Local versus Global Models for Just-In-Time Software Defect Prediction

Yang

Fan

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…at el. [2] have reported that supervised predictors did not perform outstandingly better than unsupervised ones for effort-aware just-in-time defect prediction on the basis of their experiments. Recently, Yang et al have proposed an unsupervised model and applied it to projects with rich historical bug data.…”

Section: Background and Related Workmentioning

confidence: 98%

Software Defect Prediction Via Deep Learning

Khan*¹,

Albahli²,

Albattah³

et al. 2020

IJITEE

View full text Add to dashboard Cite

Existing models on defect prediction are trained on historical limited data which has been studied from a variety of pioneering and researchers. Cross-project defect prediction, which is often reuse data from other projects, works well when the data of training models is completely sufficient to meet the project demands. However, current studies on software defect prediction require some degree of heterogeneity of metric values that does not always lead to accurate predictions. Inspired by the current research studies, this paper takes the benefit with the state-of-the-art of deep learning and random forest to perform various experiments using five different datasets. Our model is ideal for predicting of defects with 90% accuracy using 10-fold cross-validation. The achieved results show that Random Forest and Deep learning are giving more accurate predictions with compared to Bayes network and SVM on all five datasets. We also derived Deep Learning that can be competitive classifiers and provide more robust for detecting defect prediction.

show abstract

“…Then, they attempted to combine complexity metrics with more metrics such as code churn metrics and token frequency metrics [26,31,43,47,48,52,54,54,57,58,58,65,79,81]. Then, advances have been made to use unsupervised machine learning to predict bugs [25,32,36,46,75,76,77,78,80] using the similar set of complexity metrics. These approaches use the similar metrics as those in bug prediction, but do not capture the difference between vulnerable code and buggy code, which hinders the effectiveness.…”

Section: Related Workmentioning

confidence: 99%

LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Chen

et al. 2019

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Identifying potentially vulnerable locations in a code base is critical as a pre-step for effective vulnerability assessment; i.e., it can greatly help security experts put their time and effort to where it is needed most. Metric-based and pattern-based methods have been presented for identifying vulnerable code. The former relies on machine learning and cannot work well due to the severe imbalance between non-vulnerable and vulnerable code or lack of features to characterize vulnerabilities. The latter needs the prior knowledge of known vulnerabilities and can only identify similar but not new types of vulnerabilities.In this paper, we propose and implement a generic, lightweight and extensible framework, LEOPARD, to identify potentially vulnerable functions through program metrics. LEOPARD requires no prior knowledge about known vulnerabilities. It has two steps by combining two sets of systematically derived metrics. First, it uses complexity metrics to group the functions in a target application into a set of bins. Then, it uses vulnerability metrics to rank the functions in each bin and identifies the top ones as potentially vulnerable. Our experimental results on 11 real-world projects have demonstrated that, LEOPARD can cover 74.0% of vulnerable functions by identifying 20% of functions as vulnerable and outperform machine learning-based and static analysis-based techniques. We further propose three applications of LEOPARD for manual code review and fuzzing, through which we discovered 22 new bugs in real applications like PHP, radare2 and FFmpeg, and eight of them are new vulnerabilities.

show abstract

Revisiting unsupervised learning for defect prediction

Cited by 109 publications

References 55 publications

Local versus Global Models for Just-In-Time Software Defect Prediction

Local versus Global Models for Just-In-Time Software Defect Prediction

Software Defect Prediction Via Deep Learning

LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Contact Info

Product

Resources

About