McDiarmid Drift Detection Methods for Evolving Data Streams

Abstract

Increasingly, Internet of Things (IoT) domains, such as sensor networks, smart cities, and social networks, generate vast amounts of data. Such data are not only unbounded and rapidly evolving. Rather, the content thereof dynamically evolves over time, often in unforeseen ways. These variations are due to so-called concept drifts, caused by changes in the underlying data generation mechanisms. In a classification setting, concept drift causes the previously learned models to become inaccurate, unsafe and even unusable. Accordingly, concept drifts need to be detected, and handled, as soon as possible. In medical applications and emergency response settings, for example, change in behaviours should be detected in near real-time, to avoid potential loss of life. To this end, we introduce the McDiarmid Drift Detection Method (MDDM), which utilizes McDiarmid's inequality [1] in order to detect concept drift. The MDDM approach proceeds by sliding a window over prediction results, and associate window entries with weights. Higher weights are assigned to the most recent entries, in order to emphasize their importance. As instances are processed, the detection algorithm compares a weighted mean of elements inside the sliding window with the maximum weighted mean observed so far. A significant difference between the two weighted means, upper-bounded by the McDiarmid inequality, implies a concept drift. Our extensive experimentation against synthetic and real-world data streams show that our novel method outperforms the state-of-the-art. Specifically, MDDM yields shorter detection delays as well as lower false negative rates, while maintaining high classification accuracies.

ElStream: An Ensemble Learning Approach for Concept Drift Detection in Dynamic Social Big Data Stream Learning

“…They found that the focus on evolved classes may damage the results for nonevolved classes. The authors in [16] assigned higher weights to new instances for faster detection of concept drifts. In [17], the authors introduced the Fast Hoeffding Drift Detection Method that uses a sliding window and Hoeffding's inequality which enables the detection of drifts with a shorter delay.…”

An overview of unsupervised drift detection methods

“…The first class comprises detectors that use statistical tests, such as the Cumulative Sum and the Page-Hinckley Test (Page, 1954). This group includes DDM (Gama et al, 2014), EDDM (Baena-García et al, 2006), and the McDiarmid drift detection method (Pesaranghader, Viktor, & Paquet, 2018). The second class includes window-based methods, which, in general, monitor the accuracy of the classifier for instances in a window.…”

Beyond Cross-Validation—Accuracy Estimation for Incremental and Active Learning Models

“…Concept drift [24,25] describes the problem of sample distributions that change over time. This can be an abrupt event in the data stream, but it also can be a gradual, reoccurring or even virtual process [26]. Gomes et al [27] mention other possible research directions within incremental learning like anomaly detection [28], ensemble learning, recurrent neural networks and reinforcement learning.…”