Are change metrics good predictors for an evolving software product line?

Abstract

Background: Previous research on three years of early data for an Eclipse product identified some predictors of failureprone files that work well for that data set. Additionally, Eclipse has been used to explore characteristics of product line software in previous research. Aims: To assess whether change metrics are good predictors of failure-prone files over time for the family of products in the evolving Eclipse product line. Method: We repeat, to the extent possible, the decision tree portion of the prior study to assess our ability to replicate the method, and then extend it by including four more recent years of data. We compare the most prominent predictors with the previous study's results. We then look at the data for three additional Eclipse products as they evolved over time. We explore whether the set of good predictors change over time for one product and whether the set differs among products. Results: We find that change metrics are consistently good and incrementally better predictors across the evolving products in Eclipse. There is also some consistency regarding which change metrics are the best predictors. Conclusion: Change metrics are good predictors for failureprone files for the Eclipse product line. A small subset of these change metrics is fairly stable and consistent across products and releases.

An Empirical Study of Pre-release Software Faults in an Industrial Product Line

“…While others [25] have examined correlations between static code metrics and pre-release faults, we also consider change metrics. In several of the related works [11], [15], [17], change metrics were found to be better predictors of fault-proneness than static code metrics, which is consistent with our findings. Finally, throughout the literature most predictions are aimed at binary classification, i.e., classifying components as faulty or not.…”

“…Our results related to the association of the number of faults and other collected metrics support the findings of [11], [15], [17] that change metrics are more highly correlated to the number of faults in software components than static code metrics. We also found, in agreement with [2], [5], [7], [19], [25] that most faults are found in about 20% of the components.…”

“…We also found that as the SPL evolved through releases, the amount of change in the variable components remained high. More recently, in [11] we performed a further study of Eclipse as a SPL. In that work we used the J48 decision tree algorithm to analyze the effectiveness of change metrics at classifying Eclipse components as fault-prone or not.…”

Impact of restricted forward greedy feature selection technique on bug prediction

“…Krishnan et al find that change metrics are consistently good and incrementally better predictors across the evolving products in Eclipse and according to them there is also some consistency regarding which change metrics are the best predictors [7]. At the same time Menzies et al argues that static code attributes or source code metrics also have significant role in prediction of faults and identify the best predictors among source code metrics [3].…”

Toward Extended Change Types for Analyzing Software Faults

“…This study builds on the Eclipse fault data that was used by Krishnan et al to evaluate change predictors in a software product line [19]. Eclipse 2.0 and Eclipse 3.0 were selected for this study.…”