Code Churn: A Neglected Metric in Effort-Aware Just-in-Time Defect Prediction

Liu, Jinping; Zhou, Yuming; Yang, Yibiao; Lü, Hongmin; Xu, Bin

doi:10.1109/esem.2017.8

Cited by 51 publications

(50 citation statements)

References 47 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these industrial studies, only Kamei et al [17] investigated the effectiveness of effort-aware JIT defect identification approach (i.e., EALR) in an industrial setting. The effectiveness of the follow-up effort-aware JIT defect identification approaches (e.g., CBS+ [13], OneWay [7] and unsupervised approaches [23,55]) in an industrial setting has never been investigated. Furthermore, the effectiveness of supervised vs. unsupervised approaches in an industrial setting has never been explored.…”

Section: Introductionmentioning

confidence: 99%

“…This paper is the first study to investigate the effectiveness of recently-proposed effort-aware JIT defect identification approaches in an industrial setting. • We investigated the effectiveness of state-of-the-art supervised (i.e, CBS+ [13], OneWay [7] and EALR [17]) vs. unsupervised (i.e., LT [55] and Code Churn [23]) effort-aware JIT defect identification approaches on Alibaba projects. We investigated the important change-level features for effortaware JIT defect identification on Alibaba projects and their differences compared with open source projects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Yan

Xia

Fan

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Effort-aware Just-in-Time (JIT) defect identification aims at identifying defect-introducing changes just-in-time with limited code inspection effort. Such identification has two benefits compared with traditional module-level defect identification, i.e., identifying defects in a more cost-effective and efficient manner. Recently, researchers have proposed various effort-aware JIT defect identification approaches, including supervised (e.g., CBS+, OneWay) and unsupervised approaches (e.g., LT and Code Churn). The comparison of the effectiveness between such supervised and unsupervised approaches has attracted a large amount of research interest. However, the effectiveness of the recently proposed approaches and the comparison among them have never been investigated in an industrial setting. In this paper, we investigate the effectiveness of state-of-the-art effort-aware JIT defect identification approaches in an industrial setting. To that end, we conduct a case study on 14 Alibaba projects with 196,790 changes. In our case study, we investigate three aspects: (1) The effectiveness of state-of-the-art supervised (i.e., CBS+, OneWay, EALR) and unsupervised (i.e., LT and Code Churn) effortaware JIT defect identification approaches on Alibaba projects, (2) the importance of the features used in the effort-aware JIT defect identification approach, and (3) the association between projectspecific factors and the likelihood of a defective change. Moreover, we develop a tool based on the best performing approach and investigate the tool's effectiveness in a real-life setting at Alibaba.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Yan

Xia

Fan

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

show abstract

“…Shin et al [29] also showed that NML had better defect tendency prediction performance. Liu et al [31] proposed an NML based unsupervised defect prediction model (CCUM) in effortaware JIT defect prediction, and evaluated the prediction performance of CCUM under cross validation, timewise cross validation, and cross-project validation. The experimental results showed that CCUM performed better than all the prior supervised and unsupervised models.…”

Section: A Process Metricsmentioning

confidence: 99%

Which Process Metrics Are Significantly Important to Change of Defects in Evolving Projects: An Empirical Study

Jiang

Gong

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Process metrics can reflect the software development process and the code changes which are the main causes of defects. So, recently, the researches have put more emphasis on process metrics in the field of software defect prediction. For evolving projects, it is more meaningful to study whether the software module introduces or eliminates defects or not, not whether the software module is defective or defect-free. However, no such work is available in the literature focusing on the change of defect state. Discovering the factors that influence the change of defect state in the process of software development can help us to understand the causes of software defects and improve the quality of subsequent software versions. Therefore, this paper presents an extensive empirical study on which process metrics are significantly important to change of defects in evolving projects. Five process metrics of 37 versions in 12 software projects are collected. We not only analyze the class correlation values and the classification performance values among five process metrics, but also perform statistical analysis to verify whether the experimental results are of practical value. The experimental results indicate that Number of Distinct Committers plays a significantly important role in the change of defect state, especially for elimination of defects, and Number of Revisions is the second, whereas Degree of Code Modification is the last. In addition, Average Number of Modified Lines is superior to Number of Modified Lines. Based on the experimental results, some suggestions for software development and software defect prediction are also discussed.

show abstract

“…Then, they attempted to combine complexity metrics with more metrics such as code churn metrics and token frequency metrics [26,31,43,47,48,52,54,54,57,58,58,65,79,81]. Then, advances have been made to use unsupervised machine learning to predict bugs [25,32,36,46,75,76,77,78,80] using the similar set of complexity metrics. These approaches use the similar metrics as those in bug prediction, but do not capture the difference between vulnerable code and buggy code, which hinders the effectiveness.…”

Section: Related Workmentioning

confidence: 99%

LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Chen

et al. 2019

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Identifying potentially vulnerable locations in a code base is critical as a pre-step for effective vulnerability assessment; i.e., it can greatly help security experts put their time and effort to where it is needed most. Metric-based and pattern-based methods have been presented for identifying vulnerable code. The former relies on machine learning and cannot work well due to the severe imbalance between non-vulnerable and vulnerable code or lack of features to characterize vulnerabilities. The latter needs the prior knowledge of known vulnerabilities and can only identify similar but not new types of vulnerabilities.In this paper, we propose and implement a generic, lightweight and extensible framework, LEOPARD, to identify potentially vulnerable functions through program metrics. LEOPARD requires no prior knowledge about known vulnerabilities. It has two steps by combining two sets of systematically derived metrics. First, it uses complexity metrics to group the functions in a target application into a set of bins. Then, it uses vulnerability metrics to rank the functions in each bin and identifies the top ones as potentially vulnerable. Our experimental results on 11 real-world projects have demonstrated that, LEOPARD can cover 74.0% of vulnerable functions by identifying 20% of functions as vulnerable and outperform machine learning-based and static analysis-based techniques. We further propose three applications of LEOPARD for manual code review and fuzzing, through which we discovered 22 new bugs in real applications like PHP, radare2 and FFmpeg, and eight of them are new vulnerabilities.

show abstract

Code Churn: A Neglected Metric in Effort-Aware Just-in-Time Defect Prediction

Cited by 51 publications

References 47 publications

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Which Process Metrics Are Significantly Important to Change of Defects in Evolving Projects: An Empirical Study

LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Contact Info

Product

Resources

About