Heterogeneous Ensemble Combination Search Using Genetic Algorithm for Class Imbalanced Data Classification

Abstract

Classification of datasets with imbalanced sample distributions has always been a challenge. In general, a popular approach for enhancing classification performance is the construction of an ensemble of classifiers. However, the performance of an ensemble is dependent on the choice of constituent base classifiers. Therefore, we propose a genetic algorithm-based search method for finding the optimum combination from a pool of base classifiers to form a heterogeneous ensemble. The algorithm, called GA-EoC, utilises 10 fold-cross validation on training data for evaluating the quality of each candidate ensembles. In order to combine the base classifiers decision into ensemble’s output, we used the simple and widely used majority voting approach. The proposed algorithm, along with the random sub-sampling approach to balance the class distribution, has been used for classifying class-imbalanced datasets. Additionally, if a feature set was not available, we used the (α, β) − k Feature Set method to select a better subset of features for classification. We have tested GA-EoC with three benchmarking datasets from the UCI-Machine Learning repository, one Alzheimer’s disease dataset and a subset of the PubFig database of Columbia University. In general, the performance of the proposed method on the chosen datasets is robust and better than that of the constituent base classifiers and many other well-known ensembles. Based on our empirical study we claim that a genetic algorithm is a superior and reliable approach to heterogeneous ensemble construction and we expect that the proposed GA-EoC would perform consistently in other cases.

Automated Detection of Cancer Associated Genes Using a Combined Fuzzy-Rough-Set-Based F-Information and Water Swirl Algorithm of Human Gene Expression Data

“…These data are also supported by another study [14], where the repeated tuning of an MF and RS was carried out by the optimization method could achieve the dimensionality challenges and multiple-class imbalanced data for optimal classifications. The lack of previous studies with the application of WSA for gene selection and RS based on multiclass gene expression datasets, making it difficult to compare our results directly is also one of the limitations in this study.…”

“…Moreover, it was also demonstrated that finding an optimal number of genes for multiclass problems is more beneficial for diagnosis of cancer. The ensemble combinatorial search is integrated into GA [14] as a single objective GA for optimization of the ensemble technique to classify class-imbalanced datasets. Nonetheless, a single objective GA attempts to locate solutions closer to the local optimum and hence the average error is much greater than in the proposed approach, which finds global optimal solutions for the classification.…”

“…[8–11]. In addition some of the other approaches such as a decision tree [12] and ensemble classification tree [13, 14] have been used for identification of cancer-causing genes in gene expression data. Nonetheless, these approaches failed to consider the overlapping behavior of gene expression levels in uncertain situations.…”

A Comprehensive Integration Method Based on Unbalanced Data Classification Problems

“…Improvement strategy: (1) In order to ensure that the distribution of the data samples in the training data set is consistent with the overall distribution, the classification accuracy of the base classifier for a few classes of data on an unbalanced data set is improved, According to the literature [1], Firstly, the data set is changed based on the SMOTE algorithm idea and the undersampling method, and the unbalanced data is changed into relatively equalized data. Then the traditional Bagging algorithm is substituted into the base classifier to weaken the minority and majority classes.…”

“…The so-called non-equilibrium data set refers to the fact that there are far more samples of certain classes in the data set than other classes [1], in which a class with many samples is called a majority class, and a class with a few samples is called a minority class. The classification of unbalanced data sets is very common in practical applications, such as fraudulent identification of fraudulent card transactions, oil well eruption of satellite image detection, failure prediction of telecommunication equipment, medical diagnosis [2], bankruptcy prediction, radar image surveillance, financial loan management, and fraud.…”