Improved shrunken centroid classifiers for high-dimensional class-imbalanced data

Abstract

BackgroundPAM, a nearest shrunken centroid method (NSC), is a popular classification method for high-dimensional data. ALP and AHP are NSC algorithms that were proposed to improve upon PAM. The NSC methods base their classification rules on shrunken centroids; in practice the amount of shrinkage is estimated minimizing the overall cross-validated (CV) error rate.ResultsWe show that when data are class-imbalanced the three NSC classifiers are biased towards the majority class. The bias is larger when the number of variables or class-imbalance is larger and/or the differences between classes are smaller. To diminish the class-imbalance problem of the NSC classifiers we propose to estimate the amount of shrinkage by maximizing the CV geometric mean of the class-specific predictive accuracies (g-means).ConclusionsThe results obtained on simulated and real high-dimensional class-imbalanced data show that our approach outperforms the currently used strategy based on the minimization of the overall error rate when NSC classifiers are biased towards the majority class. The number of variables included in the NSC classifiers when using our approach is much smaller than with the original approach. This result is supported by experiments on simulated and real high-dimensional class-imbalanced data.

In Silico Prediction of Blood–Brain Barrier Permeability of Compounds by Machine Learning and Resampling Methods

“…At the data level, sample rescaling and resampling strategies have been used to balance data by changing the distribution of samples in different classes, including oversampling (SMOTE) and undersampling (RUS) methods. [47] At the algorithmic level, ac ost-sensitive learning approach( class weight)h as also been attempted by setting an excessive cost function to misclassification of am inority class sample. [48] In addition, an ensemble classifier combined with the resampling method such as SMOTE + ENN, which is an ovel and promising route to reduce the influence of information loss or information overfit,w as used for comparison.…”

The study of under- and over-sampling methods’ utility in analysis of highly imbalanced data on osteoporosis

“…Consequently the constructed model has low quality prediction because all objects are assigned to the dominant, negative 2 , class, regardless the value of the feature vector [19]. The bias of classification of the imbalanced data in favor of the majority class is even larger for high-dimensional data, where the number of variables greatly exceeds the number of samples [4,17,26,30,69]. And in the vast majority of medical datasets just such a situation occurs [43].…”

“…It happens that the disproportion of samples from each class is on the order of 100:1, 1 000:1 or even 10 000:1 [26]. The usage of the conventional learning methods for imbalanced data results in constructing a decision model to the majority class, which is predominant in the training set [4,26,39,41,52]. Consequently the constructed model has low quality prediction because all objects are assigned to the dominant, negative 2 , class, regardless the value of the feature vector [19].…”

“…Also a number of academic works have proposed the new approaches to classification of examined individuals as belonging to the group of patients or to identify the diagnosis factors of [4,10,23,47] contain descriptions of new methods developed to be used in the intelligent medical systems. But only some of them touch the problem of the imbalance of data.…”

Gene masking - a technique to improve accuracy for cancer classification with high dimensionality in microarray data

“…Additionally, class imbalance is an important consideration in classification of biomedical data, and there are techniques [4] which incorporate class distribution within the classification algorithm. Our approach is different in that we separate the classification from data preprocessing where we assume class imbalance is to be handled.…”