Personalized defect prediction

Abstract

Abstract-Many defect prediction techniques have been proposed. While they often take the author of the code into consideration, none of these techniques build a separate prediction model for each developer. Different developers have different coding styles, commit frequencies, and experience levels, causing different defect patterns. When the defects of different developers are combined, such differences are obscured, hurting prediction performance.This paper proposes personalized defect prediction-building a separate prediction model for each developer to predict software defects. As a proof of concept, we apply our personalized defect prediction to classify defects at the file change level. We evaluate our personalized change classification technique on six large software projects written in C and Java-the Linux kernel, PostgreSQL, Xorg, Eclipse, Lucene and Jackrabbit. Our personalized approach can discover up to 155 more bugs than the traditional change classification (210 versus 55) if developers inspect the top 20% lines of code that are predicted buggy. In addition, our approach improves the F1-score by 0.01-0.06 compared to the traditional change classification.

Deep Learning for Just-in-Time Defect Prediction

“…After we generate the advanced features, our framework next constructs a classifier (i.e., a statistical model) based on the advanced features of the training changes (Step 4). 3 In this paper, we use logistic regression [18] to build the classifier.…”

“…Defect prediction techniques are proposed to help prioritize software testing and debugging; they can recommend software components that are likely to be defective to developers. Much research has been done on defect prediction; these techniques construct predictive classification models built on features such as lines of code, code complexity and number of modified files [2], [3], [4]. Prior studies mainly focus on predicting defects at coarse granularity level, such as file, package, or module [4], [5], [6].…”

“…For example, when a team has limited resources to inspect potentially buggy lines of code, it is crucial that by manually inspecting the top percentages of lines that are likely to be buggy, developers can discover as many bugs as possible. We measure cost effectiveness as the percentage of bugs that can be discovered by inspecting the top 20% LOC based on the confidence levels that a change classification technique outputs (PofB20) [3], [1]. In addition, we also evaluate our method using the F1-score [7], [12], [16], [17], which is a summary measure that combines both precision and recall.…”

A Search-based Training Algorithm for Cost-aware Defect Prediction

“…Previous work proposed performance metrics (e.g., AUC-CE and P effort ) designed for evaluating the cost-effectiveness of prediction models [11,34,17,32,33,28]. However, the models were still built using traditional training algorithms; for example, D'Ambros et al [11] trained traditional linear regression models using the classical iteratively re-weighted least square algorithm; Rahman and Devanbu [33] used four different machine learning techniques (i.e., Logistic Regression, J48, SVM, and Naive Bayes) that were trained using the corresponding classical training algorithms.…”

Untitled

“…Examples of algorithms are logistic regression used by Zimmermann et al [14] to predict the defect proneness of classes using complexity, interaction and change metrics as predictors; Multi-Layer Perceptron (MLP), radial basis function (RBF), k-nearest neighbor (KNN), regression tree (RT), dynamic evolving neuro-fuzzy inference system (DENFIS), and Support Vector Regression (SVR) used by Elish [15] for defect prediction; Bayesian networks used by Bechta [16]; and Naive Bayes, J48, Alternative Decision Tree (ADTree), and One-R considered by Nelson et al [17]. Recently, other researchers have proposed further advanced machine learning techniques for defect prediction, such as the Multivariate Adaptive Regression Splines (MARS) [18], Personalized Change Classification (PCC) [19], Logistic Model Trees (LMT) [20], ensemble learning [21], clustering algorithms [22], and combined techniques [23]. Interestingly, Lessman et al [24] evaluated 22 classification models and showed that there is no statistical differences between the top-17 models when classifying software modules as defect prone.…”