Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach

Abstract

Difficulties of learning from nonstationary data stream are generally twofold. First, dynamically structured learning framework is required to catch up with the evolution of unstable class concepts, i.e., concept drifts. Second, imbalanced class distribution over data stream demands a mechanism to intensify the underrepresented class concepts for improved overall performance. To alleviate the challenges brought by these issues, we propose the recursive ensemble approach (REA) in this paper. To battle against the imbalanced learning problem in training data chunk received at any timestamp t, i.e., S t ; REA adaptively pushes into S t part of minority class examples received within [0, t -1] to balance its skewed class distribution. Hypotheses are then progressively developed over time for all balanced training data chunks and combined together as an ensemble classifier in a dynamically weighted manner, which therefore addresses the concept drifts issue in time. Theoretical analysis proves that REA can provide less erroneous prediction results than a comparative algorithm. Besides that, empirical study on both synthetic benchmarks and real-world data set is also applied to validate effectiveness of REA as compared with other algorithms in terms of evaluation metrics consisting of overall prediction accuracy and ROC curve.

Review of Ensemble Based Classification Algorithms for Nonstationary and Imbalanced Data

“…Chen and He propose REA [12] framework to learn on nonstationary imbalanced data streams which solves the flaw of SERA by adopting the k-nearest neighbors approach to estimate the similarity degree where each previous minority class example computes the number of minority class examples which are within its knearest neighbors of current training data chunk. REA retains all hypotheses built on each training data chunks over time and weighs them based on their classification performance on the current training data chunk.…”

Evolutionary model building under streaming data for classification tasks: opportunities and challenges

“…The diversity term can be characterized as being 'good' or 'bad' [26], a result that has a corresponding observation in EC [187]. Constructing new ensemble member Diversity of base learner [65,102] Sample stream data using Boosting versus Bagging [147,150] Identify ensemble member for replacement Age based heuristics [167] Performance based heuristics [107,171] Class imbalance Effect of sampling biases [34,55,86,186] Drift management Incremental updating of current models [107,158] Adapt voting weights [2,90,152] Shift management Outright replacement of one or more ensemble member [68,81,167] Diversity management Impact on capacity for change [26,137,165] Genet Program Evolvable Mach Under non-stationary data, it has been established that reducing the absolute value for the ensemble margin produces an equivalent increase in diversity [165]. A second open question is in regard to the method assumed for combining the outcome from multiple models under a non-stationary task.…”

“…That is to say, the distribution of classes represented in the batch can be artificially balanced with less frequently occurring classes relying on historical samples, whereas the most frequently occurring classes assume the most recent samples (e.g., [82]). Various schemes have been proposed for prioritizing retention of minor class exemplars within the batch used to construct classifiers, with k-NN algorithms frequently appearing for this purpose [34,86]. Conversely, Ditzler and Polikar [55] propose to employ oversampling or data rebalancing with ensemble methods, but not without incurring computational overheads that might limit the applicability to streaming data.…”

A reduced labeled samples (RLS) framework for classification of imbalanced concept-drifting streaming data.

“… Chapter 6 presents RLS-SSL (RLS-Semi Supervised Learning) which uses S4VM (Chen & He, 2011) as its semi-supervised learning method in conjunction with our partially labeled method to improve the quality of performance.…”

“…Hence, (Chen & He, 2011) proposed Recursive Ensemble Approach 14 (REA) which uses K-nearest neighbors as the similarity measure to address this issue.…”