Kernels for Vector-Valued Functions: A Review

Abstract

Kernel methods are among the most popular techniques in machine learning. From a frequentist/discriminative perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. From a Bayesian/generative perspective they are the key in the context of Gaussian processes, where the kernel function is also known as the covariance function. Traditionally, kernel methods have been used in supervised learning problem with scalar outputs and indeed there has been a considerable amount of work devoted to designing and learning kernels. More recently there has been an increasing interest in methods that deal with multiple outputs, motivated partly by frameworks like multitask learning. In this paper, we review different methods to design or learn valid kernel functions for multiple outputs, paying particular attention to the connection between probabilistic and functional methods.

Cross-Domain Object Recognition Via Input-Output Kernel Analysis

“…With the assumption of a multivariate distribution on class labels, we propose using MMD [18] to constrain the mismatch in the marginal distributions. Following a geometric intuition described in [31], we assume the conditional probability distributions should be similar if the marginal distributions are shifted close to each other. Therefore, we propose learning a vector-valued function in the RKHS that can give a best classification performance on the target domain data, where we put an MMD regularization on the parametric input kernel estimation and adopt output kernel estimation approach presented in [17].…”

Fast Kronecker Product Kernel Methods via Generalized Vec Trick

“…Recently, multi-task learning has been a topic of substantial research in the area of kernel methods (see e.g. [25], [26], [27]). …”

“…[29], [30], [3]), as well as a standard building block in more theoretical work concerning the development of multi-task learning methods (see e.g. [25], [31], [27]). Concerning specifically the zero-shot learning setting, Kronecker product kernel methods have been shown to outperform a variety of baseline methods in areas such as recommender systems [11], drug-target prediction [3] and image categorization [6].…”

Constrained Gaussian Process Regression for Gene-Disease Association

“…Let M m be the row index set and N n represent the column index set so that J = M×N. We use the notation MGP (φ, C N , C M ) to denote the MV-GP with mean function φ : M × N → R, row covariance function C M : M×M → R and the column covariance function C N : N × N → R. The covariance function of the MV-GP has a product structure [29], so the prior covariance between matrix entries can be decomposed as the product of the row and column covariances, C ((m, n), (m , n )) = C M (m, m )C N (n, n ).…”