Cost-Sensitive AdaBoost Algorithm for Ordinal Regression Based on Extreme Learning Machine

Abstract

In this paper, the well known stagewise additive modeling using a multiclass exponential (SAMME) boosting algorithm is extended to address problems where there exists a natural order in the targets using a cost-sensitive approach. The proposed ensemble model uses an extreme learning machine (ELM) model as a base classifier (with the Gaussian kernel and the additional regularization parameter). The closed form of the derived weighted least squares problem is provided, and it is employed to estimate analytically the parameters connecting the hidden layer to the output layer at each iteration of the boosting algorithm. Compared to the state-of-the-art boosting algorithms, in particular those using ELM as base classifier, the suggested technique does not require the generation of a new training dataset at each iteration. The adoption of the weighted least squares formulation of the problem has been presented as an unbiased and alternative approach to the already existing ELM boosting techniques. Moreover, the addition of a cost model for weighting the patterns, according to the order of the targets, enables the classifier to tackle ordinal regression problems further. The proposed method has been validated by an experimental study by comparing it with already existing ensemble methods and ELM techniques for ordinal regression, showing competitive results.

Ordinal Regression Methods: Survey and Experimental Study

“…The extension, AdaBoost.OR, proved to inherit the good properties of AdaBoost, improving both the training and test performances of existing ordinal classifiers. Another ordinal regression version of AdaBoost is proposed in [111], while in this case the adaption is based on considering a cost matrix both in pattern weighting and error updating.…”

Applying Cost-Sensitive Extreme Learning Machine and Dissimilarity Integration to Gene Expression Data Classification

“…By assigning different weights to samples following user instructions, the weighted ELM can be generalized to cost-sensitive ELM. Riccardi et al [34] worked on a cost-sensitive AdaBoost algorithm which is based on ELM. The cost-sensitive ELM is used for ordinal regression, which turns out to produce competitive results.…”

A Maximum Margin Approach for Semisupervised Ordinal Regression Clustering

“…Chu and Keerthi [2], [6] present two support vector approaches for OR by enforcing the explicit and implicit constraints on the order of hyperplanes. Besides SVM, other learning techniques [14]- [16], such as Gaussian process [17], neural networks [18], and ensemble learning [19]- [21], are also extended to solve the OR classification problems. Some other approaches [22], [23] have been presented for semisupervised OR classification.…”