A Wide Scale Classification of Class Imbalance Problem and its Solutions: A Systematic Literature Review

Abstract

In today's world, most of the data (real world) is present in imbalanced form by nature. This is because of not having efficient algorithms to put this data (i.e., generated data by billion of internetconnected devices (IoTs)) in respective format. Imbalanced data poses a great challenge to (both) data mining and machine learning algorithms. The imbalanced dataset consists of a majority class and a minority class, where the majority class takes the lead over the minority class. Generally, several standard learning algorithms assume the balanced class distribution or equal misclassification costs. If prediction is performed by these learning algorithms on imbalanced data, the accuracy will be high for majority classes, i.e., resulting in poor performance. To overcome this problem (or improving accuracy of deision/prediction-making process), data mining and machine learning researchers have addressed the problem of imbalanced data using datalevel, algorithmic level and ensemble or hybrid methods. This article presents a systematic literature review and analyze the results of more than 400 research papers published between 2002-2017 (till June 2017), resulting in a broader and elaborate investigation of the literature in this area of research. Note that extension of this article/work will contain till December 2018 research articles, which will be published in June 2019 (now these more papers/articles did not include due to no. of pages/space issues). The systematic analysis of the research literature has focus on the key role of Data Intrinsic Problems in classification, handling the imbalanced data and the techniques used to overcome the skewed distribution. Furthermore, this article reveals patterns, trends and gaps in the existing literature and discusses briefly the next generation research directions in this area.

A Novel Identification Approach Using RFECV–Optuna–XGBoost for Assessing Surrounding Rock Grade of Tunnel Boring Machine Based on Tunneling Parameters

“…Therefore, when constructing the dataset, we thoroughly considered this factor and chose the method of sample size balance. Many approaches to overcoming the class imbalance problem have been proposed [57,58]. The most commonly used methods involve the implementation of various class balancing algorithms, oversampling (such as SMOTE) [59,60], undersampling, cost-sensitive learning [61,62], or ensemble methods [63][64][65] tailored for imbalanced datasets; these are used to solve the problem of the uneven data scale distribution of different dominant lithologies in the dataset.…”

Cancer Prognosis Using Artificial Intelligence-Based Techniques

“…The most common restriction observed in our study is the insignificant quantity of data instances. Excluding the size of data, the quality of the dataset along with the cautious dimension reduction techniques [37,38] and data balancing [39,40] approaches play a significant role in effective cancer prediction results.…”

Class balancing framework for credit card fraud detection based on clustering and similarity-based selection (SBS)

“…In ALT, traditional classification algorithms are modified to deal with unbalanced datasets either by modifying cost or weights. Finally, the ELT that combines the performances of multiple classifiers to make predictions [ 15 , 16 ]. This section briefly reviews a few recent works that deal with imbalanced datasets.…”