An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift

Abstract

A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting socalled covariate shifts. In this paper, we propose an approach for semi-supervised learning algorithms that is capable of addressing this issue. Our framework also recovers some popular methods, including entropy minimization and pseudo-labeling. We provide new information-theoretical based generalization error upper bounds inspired by our novel framework. Our bounds are applicable to both general semi-supervised learning and the covariate-shift scenario. Finally, we show numerically that our method outperforms previous approaches proposed for semisupervised learning under the covariate shift.

Study on Exchange Rate Forecasting with Stacked Optimization Based on a Learning Algorithm

“…An analogy can be made with the characterization of exchange rate series based on the concept of CovariateShift [21]. This is commonly interpreted as a case in which the marginal probability distributions are different and the conditional distributions are the same, i.e., a situation in which a dataset consisting of two sets of features and labels-a training set and a test set-is defined; when the features of the training set are not the same as the features of the test set, P train (x) ̸ = P test (x), and when the labels of the training set are the same as the labels of the test set, P train y x ̸ = P test y x .…”