Time variance and defect prediction in software projects

Abstract

It is crucial for a software manager to know whether or not one can rely on a bug prediction model. A wrong prediction of the number or the location of future bugs can lead to problems in the achievement of a project's goals. In this paper we first verify the existence of variability in a bug prediction model's accuracy over time both visually and statistically. Furthermore, we explore the reasons for such a high variability over time, which includes periods of stability and variability of prediction quality, and formulate a decision procedure for evaluating prediction models before applying them. To exemplify our findings we use data from four open source projects and empirically identify various project features that influence the defect prediction quality. Specifically, we observed that a change in the number of authors editing a file and the number of defects fixed by them influence the prediction quality. Finally, we introduce an approach to estimate the accuracy of prediction models that helps a project manager decide when to rely on a prediction model. Our findings suggest that one should be aware of the periods of stability and variability of prediction quality and should use approaches such as ours to assess their models' accuracy in advance.

Developer-Related Factors in Change Prediction: An Empirical Assessment

“…This confirms previous findings by Ekanayake et al [52] on the variability of the change-proneness of classes during different stages of software evolution. As a consequence, the previous knowledge about the number of changes a class underwent is not always suitable to correctly identify change-prone classes in future versions of a software system.…”

Revisiting the Impact of Concept Drift on Just-in-Time Quality Assurance

“…By studying four open source systems, Ekanayake et al [11] investigated the notion of concept drift and its impact on defect prediction. Further study by Ekanayake et al [18] revealed that the number of authors editing software projects contributes to concept drift and results in fluctuating software quality.…”

“…Amongst the common challenges of software defect datasets tackled in literature are class imbalance [15], [16] and feature selection [17]. Ekanayake et al [18] observed that the prediction quality of models for software projects changes over time as the number of developers editing and fixing defects on the files changes.…”

“…To investigate and empirically assess the impact of concept drift on software defect quality, we revisit the study by Ekanayake et al [18] and conduct comprehensive change-level defect prediction experiments on 55 releases of 5 change-level metrics software projects using 5 prediction models and 3 evaluation measures.…”

Challenges and opportunities for software change request repositories: a systematic mapping study

“…Nevertheless, it does not necessarily indicate the importance that each topic has in CR management. Additionally, the studies [12,18,22,24,56,81,83,88,91,117,121,122,123,124] [ 69,70,71,72,73,74,75,76 ] that addressed more than one topic were classified repeatedly in each topic, that is, since Hosseini et al [19] investigated the topics of CR assignment and time to fix, their study is counted under both two topics.…”