KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift

Abstract

Abstract-Data Mining in non-stationary data streams is gaining more attention recently, especially in the context of Internet of Things and Big Data. It is a highly challenging task, since the fundamentally different types of possibly occurring drift undermine classical assumptions such as data independence or stationary distributions. Available algorithms are either struggling with certain forms of drift or require a priori knowledge in terms of a task specific setting. We propose the Self Adjusting Memory (SAM) model for the k Nearest Neighbor (kNN) algorithm since kNN constitutes a proven classifier within the streaming setting. SAM-kNN can deal with heterogeneous concept drift, i.e different drift types and rates, using biologically inspired memory models and their coordination. It can be easily applied in practice since an optimization of the meta parameters is not necessary. The basic idea is to construct dedicated models for the current and former concepts and apply them according to the demands of the given situation. An extensive evaluation on various benchmarks, consisting of artificial streams with known drift characteristics as well as real world datasets is conducted. Thereby, we explicitly add new benchmarks enabling a precise performance evaluation on multiple types of drift. The highly competitive results throughout all experiments underline the robustness of SAM-kNN as well as its capability to handle heterogeneous concept drift.

An adaptive algorithm for anomaly and novelty detection in evolving data streams

“…The Rialto dataset (Losing et al 2016) consists of images of colorful buildings next to the famous Rialto bridge in Venice, encoded in a normalized 27-dimensional RGB histogram. Images are obtained from time-lapse videos captured by a webcam with fixed position.…”

“…Moreover, existing approaches are specialized for a particular type of change (e.g., sudden, progressive, cyclic). There exist few methods which can handle different types of concept drift, such as (Losing et al 2016;Dongre and Malik 2014;Webb et al 2016;Brzezinski and Stefanowski 2014), however, most of those methods are dedicated for supervised learning problems, where the change is primarily detected by estimating a degradation in the classification performance. Other approaches such as (Kifer et al 2004;Bifet 2010) are designed to explicitly detect, in an unsupervised way, when a change happens.…”

Exploiting the stimuli encoding scheme of evolving Spiking Neural Networks for stream learning

“…• Weather dataset (Elwell & Polikar, 2011 (Losing, Hammer & Wersing, 2016). 4 equidistantly separated, squared uniform distributions are moving horizontally with constant speed.…”

Event Detection in Noisy Streaming Data with Combination of Corroborative and Probabilistic Sources

“…This approach uses several data memories, or windows of different lengths over an incoming data stream; each window has its own classifier. The SAM-KNN algorithm uses nearest neighbor approach to select the window closest to a new data sample for classification [23]. Nested windows are used in [12] to obtain multiple training sets over the same data that each exclude a region of the data space.…”