Extending Bagging for Imbalanced Data

Abstract

Various modifications of bagging for class imbalanced data are discussed. An experimental comparison of known bagging modifications shows that integrating with undersampling is more powerful than oversampling. We introduce Local-and-Over-All Balanced bagging where probability of sampling an example is tuned according to the class distribution inside its neighbourhood. Experiments indicate that this proposal is competitive to best undersampling bagging extensions.

Multi-class and feature selection extensions of Roughly Balanced Bagging for imbalanced data

“…Comparing quite low diversity of Roughly Balanced Bagging to earlier results (Błaszczyński et al 2013) we argue that RBBag is less diversified than over-bagging or SMOTE-based bagging (Wang and Yao 2009). On the other hand, RBBag is more accurate than these more diversified ensembles.…”

“…These measures are estimated with the stratified 10-fold cross-validation repeated several times to reduce the variance. All experiments were performed in the WEKA framework in which we extended the previous implementation of RBBag done by L. Idkowiak for Błaszczyński et al (2013).…”

“…Another study (Khoshgoftaar et al 2011) demonstrated that under-bagging ensembles, including RBBag, significantly outperformed best extensions of boosting and the difference was even more significant when data were noisier. Then, the comparative experiments from Błaszczyński et al (2013) showed that under-bagging ensembles, as Exactly and Roughly Balanced bagging, were significantly better than several main oversampling extensions of bagging (either using random oversampling or SMOTE) with respect to all evaluated measures. Roughly Balanced Bagging was also slightly better than Exactly Balanced one.…”

“…In the study (Khoshgoftaar et al 2011) authors applied a dozen of components. Then, 30, 50 or 70 trees were considered in Błaszczyński et al (2013). Thus, we have decided to examine more systemically other (also smaller) sizes of this ensemble and its influence on classification performance.…”

“…However, experimental comparisons (Galar et al 2011;Khoshgoftaar et al 2011) have shown that extensions of bagging work better than generalizations of boosting and other, more complex solutions. Moreover, other experiments from Błaszczyński et al (2013) have shown that undersampling modifications of bagging are significantly better than oversampling alternatives. Our recent experimental studies (Błaszczyński et al 2013;Błaszczyński and Stefanowski 2015;Stefanowski 2016b) have also demonstrated that the undersampling bagging called Roughly Balanced Bagging (RBBag) (Hido and Kashima 2009) has achieved the best prediction results comparing to other extensions of bagging as well as to modified boosting ensembles (Khoshgoftaar et al 2011).…”