RACOG and wRACOG: Two Probabilistic Oversampling Techniques

Abstract

As machine learning techniques mature and are used to tackle complex scientific problems, challenges arise such as the imbalanced class distribution problem, where one of the target class labels is under-represented in comparison with other classes. Existing oversampling approaches for addressing this problem typically do not consider the probability distribution of the minority class while synthetically generating new samples. As a result, the minority class is not well represented which leads to high misclassification error. We introduce two Gibbs sampling-based oversampling approaches, namely RACOG and wRACOG, to synthetically generating and strategically selecting new minority class samples. The Gibbs sampler uses the joint probability distribution of attributes of the data to generate new minority class samples in the form of Markov chain. While RACOG selects samples from the Markov chain based on a predefined lag, wRACOG selects those samples that have the highest probability of being misclassified by the existing learning model. We validate our approach using five UCI datasets that were carefully modified to exhibit class imbalance and one new application domain dataset with inherent extreme class imbalance. In addition, we compare the classification performance of the proposed methods with three other existing resampling techniques.

On the Performance of Oversampling Techniques for Class Imbalance Problems

“…Most of review paper and application work are based on the "classical" resampling techniques and do not take new resampling techniques into account. In this paper, we briefly review six powerful oversampling approaches, including both "classical" ones (SMOTE, ADASYN, MWMOTE) and new ones (RACOG, wRACOG, RWO-Sampling) [2,3,5,7,24]. The six reviewed oversampling techniques can be divided into two groups according to whether they consider the overall minority class distribution.…”

“…Hence, the global information of the minority samples cannot be guaranteed. In order to tackle this problem, Das et al [5] proposed RACOG (RApidy COnverging Gibbs) and wRACOG (Wrapper-based RApidy COnverging Gibbs).…”

“…RACOG selects the new minority samples from the Gibbs sampler using a predefined lag and this selection procedure does not take the usefulness of the generated samples into account. On the other hand, wRACOG considers the usefulness of the generated samples and selects those samples which have the highest probability of being misclassified by the existing learning model [5].…”

Reconstruct the Support Vectors to Improve LSSVM Sparseness for Mill Load Prediction

“…The first feature is the imbalanced distribution, which commonly exists in classification domains, such as spotting unreliable telecommunication customers, text classification, and detection of fraudulent telephone calls [26,27]. Certain solutions at the data and algorithmic levels are proposed for the classimbalance problem [28][29][30]. It is also a problem in sparse approximation.…”

Minority Oversampling in Kernel Adaptive Subspaces for Class Imbalanced Datasets

“…These additional features are called latent features and the associated feature space as latent space. RACOG and wRACOG [24] both generate instances for the minority class by considering joint probability distribution of data features. The probability distribution of the minority class is learnt using a dependence tree algorithm and the Gibbs sampler is used to generate instances from the distribution.…”