Analyzing concept drift: A case study in the financial sector

Masegosa, Andrés R.; Martínez, Ana María; Ramos-López, Darío; Langseth, Helge; Nielsen, Thomas D.; Salmerón, Antonio

doi:10.3233/ida-194515

Cited by 12 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When all instances come from the same distribution, we deal with a stationary data stream. In realworld applications, data very rarely falls under stationary assumptions [10]. It is more likely to evolve over time and form temporary concepts, being subject to concept drift [11].…”

Section: Data Stream Miningmentioning

confidence: 99%

Concept Drift Detection from Multi-Class Imbalanced Data Streams

Korycki¹,

Krawczyk²

2021

Preprint

View full text Add to dashboard Cite

Continual learning from data streams is among the most important topics in contemporary machine learning. One of the biggest challenges in this domain lies in creating algorithms that can continuously adapt to arriving data. However, previously learned knowledge may become outdated, as streams evolve over time. This phenomenon is known as concept drift and must be detected to facilitate efficient adaptation of the learning model. While there exists a plethora of drift detectors, all of them assume that we are dealing with roughly balanced classes. In the case of imbalanced data streams, those detectors will be biased towards the majority classes, ignoring changes happening in the minority ones. Furthermore, class imbalance may evolve over time and classes may change their roles (majority becoming minority and vice versa). This is especially challenging in the multi-class setting, where relationships among classes become complex. In this paper, we propose a detailed taxonomy of challenges posed by concept drift in multi-class imbalanced data streams, as well as a novel trainable concept drift detector based on Restricted Boltzmann Machine. It is capable of monitoring multiple classes at once and using reconstruction error to detect changes in each of them independently. Our detector utilizes a skew-insensitive loss function that allows it to handle multiple imbalanced distributions. Due to its trainable nature, it is capable of following changes in a stream and evolving class roles, as well as it can deal with local concept drift occurring in minority classes. Extensive experimental study on multi-class drifting data streams, enriched with a detailed analysis of the impact of local drifts and changing imbalance ratios, confirms the high efficacy of our approach.

show abstract

Section: Data Stream Miningmentioning

confidence: 99%

Concept Drift Detection from Multi-Class Imbalanced Data Streams

Korycki¹,

Krawczyk²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…When all instances arriving over time originate from the same distribution, we deal with a stationary stream that requires only incremental learning and no adaptation. However, in real-world applications data very rarely falls under stationary assumptions (Masegosa et al., 2020 ). It is more likely to evolve over time and form temporary concepts, being subject to concept drift (Lu et al., 2019 ).…”

Section: Data Stream Miningmentioning

confidence: 99%

Adversarial concept drift detection under poisoning attacks for robust data stream mining

Korycki

Krawczyk

2022

Mach Learn

View full text Add to dashboard Cite

Continuous learning from streaming data is among the most challenging topics in the contemporary machine learning. In this domain, learning algorithms must not only be able to handle massive volume of rapidly arriving data, but also adapt themselves to potential emerging changes. The phenomenon of evolving nature of data streams is known as concept drift. While there is a plethora of methods designed for detecting its occurrence, all of them assume that the drift is connected with underlying changes in the source of data. However, one must consider the possibility of a malicious injection of false data that simulates a concept drift. This adversarial setting assumes a poisoning attack that may be conducted in order to damage the underlying classification system by forcing an adaptation to false data. Existing drift detectors are not capable of differentiating between real and adversarial concept drift. In this paper, we propose a framework for robust concept drift detection in the presence of adversarial and poisoning attacks. We introduce the taxonomy for two types of adversarial concept drifts, as well as a robust trainable drift detector. It is based on the augmented restricted Boltzmann machine with improved gradient computation and energy function. We also introduce Relative Loss of Robustness—a novel measure for evaluating the performance of concept drift detectors under poisoning attacks. Extensive computational experiments, conducted on both fully and sparsely labeled data streams, prove the high robustness and efficacy of the proposed drift detection framework in adversarial scenarios.

show abstract

“…Concept drift can arise due to a wide variety of reasons. For example, the inance industry faced turbulent changes as the inancial crisis of 2008 was unfolding, and if advanced detection techniques were employed they could have provided additional insights into the ongoing crisis, as explained by Masegosa et al [88]. Changes in data can also be caused by an inability to avoid luctuations in the data collection procedure, as described in the paper by Langenkämper et al [73] which studies the efects of slight changes in marine images on deep learning models' performance.…”

Section: Updatingmentioning

confidence: 99%

Challenges in Deploying Machine Learning: A Survey of Case Studies

Urma²,

2022

View full text Add to dashboard Cite

In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process. The goal of this paper is to lay out a research agenda to explore approaches addressing these challenges.

show abstract

Analyzing concept drift: A case study in the financial sector

Cited by 12 publications

References 20 publications

Concept Drift Detection from Multi-Class Imbalanced Data Streams

Concept Drift Detection from Multi-Class Imbalanced Data Streams

Adversarial concept drift detection under poisoning attacks for robust data stream mining

Challenges in Deploying Machine Learning: A Survey of Case Studies

Contact Info

Product

Resources

About