SAE: Social Adaptive Ensemble classifier for data streams

Abstract

This work encompasses the development of a new ensemble classifier that uses a Social Network abstraction for Data Stream Classification, namely the Social Adaptive Ensemble (SAE). In the context of data stream classification, concept drift is considered one of the most difficult and important issues to be addressed. Ensemble classifiers can be successfully applied to data streams as long as the ensemble efficiently adapts itself in the occurrence of a concept drift. SAE algorithm inherits strategies from other ensemble methods, such as Online Bagging [4] and DWM [2], and merge these with the notion of connectivity between similar classifiers w.r.t. their individual predictions. The relational data obtained through measuring similarities between classifiers is used to arrange ensemble members in a social network structure that allows us to identify subgroups (subnetworks) of highly similar classifiers. Being able to identify similar classifiers allows us to implement a combination strategy that first combines predictions within similar classifiers and later combines these into the final prediction. Moreover, this combination strategy assigns more weight to recently added classifiers predictions during concept drifts, since these are dissimilar to all other existing classifiers. The similarity between classifiers is also used to identify and remove redundant classifiers. This effectively saves systems resources and sometimes improves accuracy. We present empirical experiments with synthetic data streams containing abrupt, gradual and no drift showing that SAE is a valid option for stream classification, especially when data stream characteristics (e.g. presence of abrupt drifts) are previously unknown and system resources, such as CPU time and memory space, are a concern.

Pairwise combination of classifiers for ensemble learning on data streams

“…The original Social Adaptive Ensemble (SAE) [12] algorithm and its improved version (SAE2) [13] are focused on three aspects of ensemble classifiers in a data stream scenario: diversity, combination and adaptation. SAE and SAE2, through different means, attempt to quantify diversity, and combine individual predictions in a way that similar classifiers predictions are grouped.…”

“…Other works usually focus on building highly diverse ensembles [5,12,13,22], although it is always expected some degree of overlap between classifiers. The voting strategies presented here are aimed at using these overlaps to support ensemble prediction.…”

“…Ensemble classifiers achieve high accuracy through the combination of a diverse set of component classifiers, such that (ideally) incorrect predictions are obfuscated while correct predictions are highlighted. In a data stream scenario, ensemble classifiers also have the advantageous characteristic of being flexible, i.e., it is possible to replace (or remove) component classifiers based on drift detector algorithms [5,6] or other methods [8,12,13,18]. Even though there is not a solid proof of a strong correlation between accuracy and diversity [19,20], it is possible to verify that a set of homogeneous classifiers that always decide for the same labels cannot achieve better (or worse) results than one of them alone.…”

“…It is important to note that diversity is favorable as long as the individual decisions are properly combined. Existing ensemble classifiers for data streams use various techniques for combining predictions, such as majority vote, weighted majority [21] and other methods [2,8,13] that attempt to incorporate characteristics, mainly to account for concept drifts, that benefit classifiers adapted to the current concept over others.…”

SFNClassifier

“…The hypothesis is that prominent classifiers are more likely to establish new connections hence improving its centrality. As in DWM [16] and SAE [13], our proposal is based in a period size parameter p that determines how many instances will be evaluated before a network update takes place.…”