Mining concept-drifting data streams using ensemble classifiers

Abstract

Recently, mining data streams with concept drifts for actionable insights has become an important and challenging task for a wide range of applications including credit card fraud protection, target marketing, network intrusion detection, etc. Conventional knowledge discovery tools are facing two challenges, the overwhelming volume of the streaming data, and the concept drifts. In this paper, we propose a general framework for mining concept-drifting data streams using weighted ensemble classifiers. We train an ensemble of classification models, such as C4.5, RIPPER, naive Bayesian, etc., from sequential chunks of the data stream. The classifiers in the ensemble are judiciously weighted based on their expected classification accuracy on the test data under the time-evolving environment. Thus, the ensemble approach improves both the efficiency in learning the model and the accuracy in performing classification. Our empirical study shows that the proposed methods have substantial advantage over single-classifier approaches in prediction accuracy, and the ensemble framework is effective for a variety of classification models.

An Efficient Concept Drift Detection Method for Streaming Data under Limited Labeling

“…In a concept-drifting data stream, a new member of the ensemble family is built on a chunk of recent data samples and an outdated member is removed. By assigning weights to ensemble members depending on the estimated error rate, concept drift can be dealt with [15]. In [16], two ensemble methods, bagging by ADWIN (ADaptive WINdowing) [14] and bagging by adaptive-size Hoeffding trees, are proposed.…”

Multivariate Stream Data Classification Using Simple Text Classifiers

“…Multivariate time series data classification methods were studied in [4,5,6,7,8], including On-demand Classifier [4], HMM (Hidden Markov Models) [5], RNN (Recurrent Neural Network), Dynamic Time Warping [5], weighted ensemble classifier [6] and SAX [7]. These methods involve large numbers of parameters and complex preprocessing step that need to be tuned.…”

Classification and Novel Class Detection in Data Streams with Active Mining

“…So, supervised classification techniques suffer from the scarcity of labeled data for learning, resulting in a poorly built classifier. Most existing data stream classification techniques address only the infinite length, and concept-drift problems [1][2][3]. Our previous work MineClass [4] addresses the concept-evolution problem in addition to the infinite length and conceptdrift problems.…”

“…In order to cope with concept-drift, a classifier must be continuously updated to be consistent with the most recent concept. ActMiner applies a hybrid batch-incremental process [2,5] to solve the infinite length and concept-drift problems. It divides the data stream into equal sized chunks and trains a classification model from each chunk.…”

“…The single-model approaches in group i) apply incremental learning and adapt themselves to the most recent concept by continuously updating the current model to accommodate concept drift [1,3,6]. Ensemble techniques [2,5] maintain an ensemble of models, and use ensemble voting to classify unlabeled instances. These techniques address the infinite length problem by keeping a fixed-size ensemble, and address the concept-drift problem by updating the ensemble with newer models.…”