Semi-supervised learning for ordinal Kernel Discriminant Analysis

Abstract

Ordinal classification considers those classification problems where the labels of the variable to predict follow a given order. Naturally, labelled data is scarce or difficult to obtain in this type of problems because, in many cases, ordinal labels are given by an user or expert (e.g. in recommendation systems). Firstly, this paper develops a new strategy for ordinal classification where both labelled and unlabelled data are used in the model construction step (a scheme which is referred to as semi-supervised learning). More specifically, the ordinal version of kernel discriminant learning is extended for this setting considering the neighbourhood information of unlabelled data, which is proposed to be computed in the feature space induced by the kernel function. Secondly, a new method for semi-supervised kernel learning is devised in the context of ordinal classification, which is combined with our developed classification strategy to optimise the kernel parameters. The experiments conducted compare 6 different approaches for semi-supervised learning in the context of ordinal classification in a battery of 30 datasets, showing 1) the good synergy of the ordinal version of discriminant analysis and the use of unlabelled data and 2) the advantage of computing distances in the feature space induced by the kernel function.

Incremental sparse Bayesian ordinal regression

“…Analyzing Table 5, we notice that the sparse Bayes based SBOR and ISBOR employ much smaller numbers of training samples to make predictions than the SVM-based SVOR and ISVOR. 8 Among the seven benchmark datasets, ISBOR wins 5 times and SBOR wins 2 times, which supports our claim that ISBOR is a parsimonious ordinal regression algorithm and can make effective predictions based on a small subset of the training set. This finding answers RQ3 on sparseness.…”

“…The task of modeling ordinal data has attracted attention in various areas, including computer vision [1,2], information retrieval [3], recommender systems [4] and machine learning [5,6,7,8,9]. Because of the explicit or implicit relationship between labels, simple regression or multi-classification algorithms may fail to find optimal decision boundaries, which motivates the development of dedicated methods.…”

A new approach: semisupervised ordinal classification

“…Semi-supervised ordinal regression (SSOR) has very little coverage in the literature, only several detailed analyses [18][19][20][21] has been performed. Table 1 shows the comparison of our SSOC study with the existing SSOR studies.…”

“…First, they performed the prediction of numeric target values, while we focus on the classification of categorical target values. Second, they used different methods such as kernel discriminant learning [18], empirical risk minimization [19], max-coupled learning [20], and Gaussian processes [21]; whereas we utilized the ordinal binary decomposition method. Third, since their target values are numeric they used different evaluation metrics such as mean absolute error (MAE), mean squared error (MSE), and mean zero-one error (MZE), whereas we tested the performance with accuracy and F-score metrics.…”

A Novel Variational Autoencoder with Applications to Generative Modelling, Classification, and Ordinal Regression

“…Also variational Gaussian process auto-encoders were developed in Eleftheriadis et al (2016) for ordinal prediction of facial action units. Other techniques include kernel discriminant analysis with an additional order forcing term (Pérez-Ortiz et al, 2016).…”