A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction

Abstract

In this paper we present a comparative analysis of the predictive power of two different sets of metrics for defect prediction. We choose one set of product related and one set of process related software metrics and use them for classifying Java files of the Eclipse project as defective respective defect-free. Classification models are built using three common machine learners: logistic regression, Naïve Bayes, and decision trees. To allow different costs for prediction errors we perform cost-sensitive classification, which proves to be very successful: >75% percentage of correctly classified files, a recall of >80%, and a false positive rate <30%. Results indicate that for the Eclipse data, process metrics are more efficient defect predictors than code metrics.

A Framework for Defect Prediction in Specific Software Project Contexts

“…An N/A reveals that there is no information that contains the definition of an expected term in the study. [14] Goal is implicitly described Questions proposed with respective null hypotheses Implicit variables specifications to predict module defect proneness Li et al [9] Goal is implicitly described…”

“…Thus, we assert that the first phase in our framework which consists of goal definition, research questions and hypotheses formulation, and variable specifications is a common practice in conducting defect prediction with different levels of detail and presentation. [11] Product and process 16 modeling N/A Average relative Li et al [11] Product [14] Cross validation with different releases with low performance results N/A Li et al [9] Constructed model were used to predict a certain period of defect growth per release…”

“…There are several reasons why many studies did not report the external validation and robustness analysis of constructed prediction model such as the availability of new observation data [22] and external validation results which signify poor performance of the model [14] for which many of the authors do not wish to report. However from practitioners' perspective such conditions should be addressed properly by data collection process refinement and model calibrations until the model can be proven for its usefulness for prediction in particular context.…”

“…However, Koru et al [8] advise that in practice, the most appropriate prediction method has to be selected for the current project context and the type of defect pattern to be predicted. Thereby, a good defect prediction model has to be constructed using a set of predictor variables that represents the actual measures of the software product and process [14,15,26]. Furthermore, several measures to evaluate the quality of a prediction are recommended, e.g.…”

Cost Effectiveness of Software Defect Prediction in an Industrial Project

“…Since prediction results are categorical (faulty or not-faulty), we decided to test classifiers often used in software defect prediction [7,14,25,38], which are available in the basic package of KNIME:…”