Graph-Based Approaches for Over-Sampling in the Context of Ordinal Regression

Abstract

The classification of patterns into naturally ordered labels is referred to as ordinal regression or ordinal classification. Usually, this classification setting is by nature highly imbalanced, because there are classes in the problem that are a priori more probable than others. Although standard over-sampling methods can improve the classification of minority classes in ordinal classification, they tend to introduce severe errors in terms of the ordinal label scale, given that they do not take the ordering into account. A specific ordinal over-sampling method is developed in this paper for the first time in order to improve the performance of machine learning classifiers. The method proposed includes ordinal information by approaching over-sampling from a graph-based perspective. The results presented in this paper show the good synergy of a popular ordinal regression method (a reformulation of support vector machines) with the graph-based proposed algorithms, and the possibility of improving both the classification and the ordering of minority classes. A cost-sensitive version of the ordinal regression method is also introduced and compared with the over-sampling proposals, showing in general lower performance for minority classes.

Machine Learning with Applications in Breast Cancer Diagnosis and Prognosis

“…The early findings showed results to the contrary; in 1998, Fogel et al compared ANN with linear discriminant analysis of the radiographic features of masses and patient's age in 139 suspicious masses to detect whether they developed breast cancer and demonstrated the superiority of latter [22]. Lately however with the maturity and improvement of the ML algorithms, and the increasing quantity and complexity of the data, results show ML approaches have better classification accuracy [12,14,15,[23][24][25]. In 2004, two ML classification methods (DT and ANN) were compared with a statistical method (linear regression) to predict the breast cancer survival using a large dataset which has more than 200, 000 cases and demonstrated that ML methods could be a promising classification method for practical use.…”

Cluster-Based Minority Over-Sampling for Imbalanced Datasets

“…Ordinal graph-based over-sampling (OGO-SP) [7] uses the k-NN algorithm to generate a graph between three ordinal classes in which edges and vertices respectively represent nearest neighbors and data samples. The k-NN graph is generated within the same class and inter-classes.…”

“…⃝ 2016 The Institute of Electronics, Information and Communication Engineers niques for over sampling methods [2]- [7] that are developed based on SMOTE. To evaluate performance of TRIM as a pre-processing method, we combine it with SMOTE and BSMOTE (an extended version of SMOTE).…”

“…The advantage of synthetic over-sampling is its generalization ability to produce a decision region larger and less specific toward the original dataset. Several synthetic over-sampling techniques [2]- [7] have been proposed. However, almost all of them suffer from the over-generalization problem where synthetic minority class samples are generated in the majority class region.…”

Multiple Ordinal Regression by Maximizing the Sum of Margins

“…Other recent ordinal regression approaches provide extensions for ensemble learning [29], [11], sampling problems [30], and semi-supervised learning [37]. [29] proposes an ensemble learning for ordinal regression by extracting multiple projection-based two class classifiers and three class ordinal regressors.…”