Multi-Label Punitive kNN with Self-Adjusting Memory for Drifting Data Streams

Abstract

In multi-label learning, data may simultaneously belong to more than one class. When multi-label data arrives as a stream, the challenges associated with multi-label learning are joined by those of data stream mining, including the need for algorithms that are fast and flexible, able to match both the speed and evolving nature of the stream. This article presents a punitive k nearest neighbors algorithm with a self-adjusting memory (MLSAMPkNN) for multi-label, drifting data streams. The memory adjusts in size to contain only the current concept and a novel punitive system identifies and penalizes errant data examples early, removing them from the window. By retaining and using only data that are both current and beneficial, MLSAMPkNN is able to adapt quickly and efficiently to changes within the data stream while still maintaining a low computational complexity. Additionally, the punitive removal mechanism offers increased robustness to various data-level difficulties present in data streams, such as class imbalance and noise. The experimental study compares the proposal to 24 algorithms using 30 real-world and 15 artificial multi-label data streams on six multi-label metrics, evaluation time, and memory consumption. The superior performance of the proposed method is validated through non-parametric statistical analysis, proving both high accuracy and low time complexity. MLSAMPkNN is a versatile classifier, capable of returning excellent performance in diverse stream scenarios.

A Survey on Multi-Label Data Stream Classification

“…Contrary to active learning, the passive learning adopts an adaption mechanism, for example, [112] used a sophisticated parameterized windowing technique to phase out old examples over time instead of detecting drifts. The use of two self-adjust-memories in [132] allows ML-SAM-kNN to adapt to various of drifts, such as gradual, recurring drift, etc.…”

Markov Boundary Learning With Streaming Data for Supervised Classification

“…There are several commonly used methods for data stream classification, and each group has a few excellent algorithms. For example, the SAM [16], ML-SAM-kNN [24] and ML-SAMPkNN [25] are typical instance-based learning methods. For ensemble learning methods, the most common approaches are Bagging and Bosting, which are firstly proposed by Oza and Russell [20], like OBA [19] and LB [4].…”

Introductory Chapter: Data Streams and Online Learning in Social Media

“…Some real-world problems are characterized for having instances simultaneously categorized into multiple labels. This problem is known as multi-label learning [19][20]. The complexity of correctly classifying the instance increases with the size of the output space.…”

“…Therefore, it is more difficult to detect and adapt to concept drift. Authors have proposed solutions for multi-label data streams, including self-adjusting windows to identify the more accurate and most recent subset of instances in a sliding window [19]. Moreover, punitive systems have shown that penalizing instances leading to erroneous label predictions and early removing them from the window increase the overall accuracy of the classifier [21].…”