Concept drift region identification via competence-based discrepancy distribution estimation

Fan, Dashan; Lu, Jie; Li, Kan; Zhang, Guangquan

doi:10.1109/iske.2017.8258734

Cited by 11 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dong et al in [18] proved and showed that a well-trained defect-prediction model may output inaccurate results if the data distribution changes over time. They also proved that training a prediction model becomes more difficult in such scenarios.…”

Section: Concept-drift Handlingmentioning

confidence: 99%

“…The stable learner predicts (line 14), whereas the reactive one performs prediction (line 15) using the data samples in the currently processing window. The number of places in the current window where predictions of the reactive learner are correct whereas that of the stable learner are incorrect are computed (lines [16][17][18][19][20]. This count is then compared with the user-supplied threshold value for detecting drift.…”

Section: Proposed Approach Based On Plmentioning

confidence: 99%

See 1 more Smart Citation

A Paired Learner-Based Approach for Concept Drift Detection and Adaptation in Software Defect Prediction

2021

View full text Add to dashboard Cite

The early and accurate prediction of defects helps in testing software and therefore leads to an overall higher-quality product. Due to drift in software defect data, prediction model performances may degrade over time. Very few earlier works have investigated the significance of concept drift (CD) in software-defect prediction (SDP). Their results have shown that CD is present in software defect data and tha it has a significant impact on the performance of defect prediction. Motivated from this observation, this paper presents a paired learner-based drift detection and adaptation approach in SDP that dynamically adapts the varying concepts by updating one of the learners in pair. For a given defect dataset, a subset of data modules is analyzed at a time by both learners based on their learning experience from the past. A difference in accuracies of the two is used to detect drift in the data. We perform an evaluation of the presented study using defect datasets collected from the SEACraft and PROMISE data repositories. The experimentation results show that the presented approach successfully detects the concept drift points and performs better compared to existing methods, as is evident from the comparative analysis performed using various performance parameters such as number of drift points, ROC-AUC score, accuracy, and statistical analysis using Wilcoxon signed rank test.

show abstract

Section: Concept-drift Handlingmentioning

confidence: 99%

Section: Proposed Approach Based On Plmentioning

confidence: 99%

A Paired Learner-Based Approach for Concept Drift Detection and Adaptation in Software Defect Prediction

2021

View full text Add to dashboard Cite

show abstract

“…This refers to the phenomenon where the statistical properties of a data stream change over time [9]. If CD is not detected, the model may become less accurate over time, leading to false positives or false negatives [10]. Therefore, there is a need for concept drift detection (CDD) methods that can identify when CD is occurring and adjust the model accordingly.…”

Section: Introductionmentioning

confidence: 99%

Interpretation Conclusion Stability of Software Defect Prediction over Time

chitsazian,

Kashi,

Nikanjam

2024

Preprint

View full text Add to dashboard Cite

Model instability refers to where a machine learning model trained on historical data becomes less reliable over time due to Concept Drift (CD). CD refers to the phenomenon where the underlying data distribution changes over time. In this paper, we proposed a method for predicting CD in evolving software through the identification of inconsistencies in the instance interpretation over time for the first time. To this end, we obtained the instance interpretation vector for each newly created commit sample by developers over time. Wherever there is a significant difference in statistical distribution between the interpreted sample and previously ones, it is identified as CD. To evaluate our proposed method, we have conducted a comparison of the method's results with those of the baseline method. The baseline method locates CD points by monitoring the Error Rate (ER) over time. In the baseline method, CD is identified whenever there is a significant rise in the ER. In order to extend the evaluation of the proposed method, we have obtained the CD points by the baseline method based on monitoring additional efficiency measures over time besides the ER. Furthermore, this paper presents an experimental study to investigate the discovery of CD over time using the proposed method by taking into account resampled datasets for the first time. The results of our study conducted on 20 known datasets indicated that the model's instability over time can be predicted with a high degree of accuracy without requiring the labeling of newly entered data.

show abstract

“…CD can have a significant impact on the accuracy of JIT-SDP models. If CD is not detected and accounted for, the model may become less accurate over time, leading to false positives or false negatives (Dong, Lu et al 2017). By detecting and addressing concept drift in JIT-SDP models, organizations can improve their ability to identify and address software defects before they become critical issues.…”

Section: Introductionmentioning

confidence: 99%

Concept Drift Detection in Just-in-Time Software Defect Prediction Using Model Interpretation

chitsazian¹,

Kashi²,

Nikanjam³

2023

Preprint

View full text Add to dashboard Cite

Concept drift region identification via competence-based discrepancy distribution estimation

Cited by 11 publications

References 17 publications

A Paired Learner-Based Approach for Concept Drift Detection and Adaptation in Software Defect Prediction

A Paired Learner-Based Approach for Concept Drift Detection and Adaptation in Software Defect Prediction

Interpretation Conclusion Stability of Software Defect Prediction over Time

Concept Drift Detection in Just-in-Time Software Defect Prediction Using Model Interpretation

Contact Info

Product

Resources

About