Concept drift from 1980 to 2020: a comprehensive bibliometric analysis with future research insight

Babüroğlu, Elif Selen; Durmuşoğlu, Alptekin; Dereli, Türkay

doi:10.1007/s12530-023-09503-2

Cited by 1 publication

(1 citation statement)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The concept of dataset shift or drift was first identified in the data-stream mining community [7,23,24]. Detecting the dataset drift is still very challenging in non-stationary environments, as in these environments, the data change over time [25].…”

Section: Concept Driftmentioning

confidence: 99%

Cross-Version Software Defect Prediction Considering Concept Drift and Chronological Splitting

Kabir,

Rehman,

Islam

et al. 2023

Symmetry

View full text Add to dashboard Cite

Concept drift (CD) refers to a phenomenon where the data distribution within datasets changes over time, and this can have adverse effects on the performance of prediction models in software engineering (SE), including those used for tasks like cost estimation and defect prediction. Detecting CD in SE datasets is difficult, but important, because it identifies the need for retraining prediction models and in turn improves their performance. If the concept drift is caused by symmetric changes in the data distribution, the model adaptation process might need to account for this symmetry to maintain accurate predictions. This paper explores the impact of CD within the context of cross-version defect prediction (CVDP), aiming to enhance the reliability of prediction performance and to make the data more symmetric. A concept drift detection (CDD) approach is further proposed to identify data distributions that change over software versions. The proposed CDD framework consists of three stages: (i) data pre-processing for CD detection; (ii) notification of CD by triggering one of the three flags (i.e., CD, warning, and control); and (iii) providing guidance on when to update an existing model. Several experiments on 30 versions of seven software projects reveal the value of the proposed CDD. Some of the key findings of the proposed work include: (i) An exponential increase in the error-rate across different software versions is associated with CD. (ii) A moving-window approach to train defect prediction models on chronologically ordered defect data results in better CD detection than using all historical data with a large effect size (δ≥0.427).

show abstract