Fine-grained just-in-time defect prediction

Abstract

Defect prediction models focus on identifying defect-prone code elements, for example to allow practitioners to allocate testing resources on specific subsystems and to provide assistance during code reviews. While the research community has been highly active in proposing metrics and methods to predict defects on long-term periods (i.e., at release time), a recent trend is represented by the so-called short-term defect prediction (i.e., at commit-level). Indeed, this strategy represents an effective alternative in terms of effort required to inspect files likely affected by defects. Nevertheless, the granularity considered by such models might be still too coarse. Indeed, existing commit-level models highlight an entire commit as defective even in cases where only specific files actually contain defects. In this paper, we first investigate to what extent commits are partially defective; then, we propose a novel fine-grained just-in-time defect prediction model to predict the specific files, contained in a commit, that are defective. Finally, we evaluate our model in terms of (i) performance and (ii) the extent to which it decreases the effort required to diagnose a defect. Our study highlights that: (1) defective commits are frequently composed of a mixture of defective and nondefective files, (2) our fine-grained model can accurately predict defective files with an AUC-ROC up to 82% and (3) our model would allow practitioners to save inspection efforts with respect to standard just-in-time techniques.

Predicting Defective Lines Using a Model-Agnostic Technique

“…Such code changes (or commits) must undergo rigorous SQA activities (e.g., Continuous Integration tests and code review) prior to merge into the main branch (e.g., a master branch) [24]. Since these commit-level SQA activities are time-consuming, Just-In-Time defect prediction has been proposed to support developers by prioritizing their limited SQA effort on the most risky code changes that will introduce software defects during the development cycle (i.e., pre-release defects) [45,61]. Nevertheless, JIT defect prediction only early detects defect-inducing changes, rather than post-release defects (i.e., the areas of code that are likely to be defective after a release).…”

“…The cost-effectiveness of the SQA resource prioritization heavily relies on the granularity levels of defect prediction. Prior studies argued that prioritizing software modules at the finer granularity is more cost-effective [28,43,61]. For example, Kamei et al [43] found that the file-level defect prediction is more effective than the package-level defect prediction.…”

“…Motivation. One of the key benefits of defect prediction is to help developers perform a cost-effective SQA activity by priortizing defect-prone files in order to uncover maximal defects with minimal effort [28,43,53,61]. In other words, an effective prioritization should rank defective lines to the top in order to help developers find more defects given the limited amount of effort.…”

Process metrics for software defect prediction in object‐oriented programs

“…In view of the software change, the researchers have proposed the change‐level defect prediction, which is known as just‐in‐time defect prediction [23, 24]. Unlike the traditional defect prediction methods, the change‐level defect prediction methods regard a change as an instance for training and predict whether new software changes introduce changes or not.…”

Local versus Global Models for Just-In-Time Software Defect Prediction

“…Considering a commit in the software development may be partially defective, which may involve defective files and nondefective files. To this end, Pascarella et al [23] proposed a fine-grained JIT-SDP model to predict buggy files in a commit based on ten open-source projects. Experimental results show that 43% defective commits are mixed by buggy and clean resources, and their method can obtain 82% AUC-ROC to detect defective files in a commit.…”