Learning under Concept Drift: an Overview

Žliobaitė, Indrė

doi:10.48550/arxiv.1010.4784

Cited by 27 publications

(33 citation statements)

References 114 publications

(165 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, studying the problem at this level of generality has led to a number of difficulties in creating a unified language and objective [41,14], an issue we circumvent by assuming that the population distribution is determined by the deployed classifier. Importantly, this line of work also discusses the importance of retraining [42,14]. However, it stops short of discussing the need for stability or analyzing the long-term behavior of retraining.…”

Section: Related Workmentioning

confidence: 99%

Performative Prediction

Perdomo¹,

Zrnic²,

Mendler-Dünner³

et al. 2020

Preprint

View full text Add to dashboard Cite

When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far been neglected in supervised learning. When ignored, performativity surfaces as undesirable distribution shift, routinely addressed with retraining.We develop a risk minimization framework for performative prediction bringing together concepts from statistics, game theory, and causality. A conceptual novelty is an equilibrium notion we call performative stability. Performative stability implies that the predictions are calibrated not against past outcomes, but against the future outcomes that manifest from acting on the prediction. Our main results are necessary and sufficient conditions for the convergence of retraining to a performatively stable point of nearly minimal loss.In full generality, performative prediction strictly subsumes the setting known as strategic classification. We thus also give the first sufficient conditions for retraining to overcome strategic feedback effects. * Equal contribution.

show abstract

Section: Related Workmentioning

confidence: 99%

Performative Prediction

Perdomo¹,

Zrnic²,

Mendler-Dünner³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In some cases, concept drifts in data streams should be detected promptly [104]. Žliobaitė provides a framework for thinking about decision points when addressing concept drift [105].…”

Section: Dealing With the History Of Experimentsmentioning

confidence: 99%

A software engineering perspective on engineering machine learning systems: State of the art and challenges

Giray¹

2021

Journal of Systems and Software

100

View full text Add to dashboard Cite

“…However, concepts are often not stable in the real world but change over time [9]. Concept drifts are usually classified into the following categories [15]: Sudden concept drift where the data changes very quickly (e.g. sudden machine failures), incremental and gradual concept drift (e.g.…”

Section: Concept Driftmentioning

confidence: 99%

Switching Scheme: A Novel Approach for Handling Incremental Concept Drift in Real-World Data Sets

Baier¹,

Kellner²,

Kühl³

et al. 2020

Preprint

View full text Add to dashboard Cite

Machine learning models nowadays play a crucial role for many applications in business and industry. However, models only start adding value as soon as they are deployed into production. One challenge of deployed models is the effect of changing data over time, which is often described with the term concept drift. Due to their nature, concept drifts can severely affect the prediction performance of a machine learning system.In this work, we analyze the effects of concept drift in the context of a real-world data set. For efficient concept drift handling, we introduce the switching scheme which combines the two principles of retraining and updating of a machine learning model. Furthermore, we systematically analyze existing regular adaptation as well as triggered adaptation strategies. The switching scheme is instantiated on New York City taxi data, which is heavily influenced by changing demand patterns over time. We can show that the switching scheme outperforms all other baselines and delivers promising prediction results.

show abstract

Learning under Concept Drift: an Overview

Cited by 27 publications

References 114 publications

Performative Prediction

Performative Prediction

A software engineering perspective on engineering machine learning systems: State of the art and challenges

Switching Scheme: A Novel Approach for Handling Incremental Concept Drift in Real-World Data Sets

Contact Info

Product

Resources

About