Bayesian Multitask Classification With Gaussian Process Priors

Abstract

We present a novel approach to multitask learning in classification problems based on Gaussian process (GP) classification. The method extends previous work on multitask GP regression, constraining the overall covariance (across tasks and data points) to factorize as a Kronecker product. Fully Bayesian inference is possible but time consuming using sampling techniques. We propose approximations based on the popular variational Bayes and expectation propagation frameworks, showing that they both achieve excellent accuracy when compared to Gibbs sampling, in a fraction of time. We present results on a toy dataset and two real datasets, showing improved performance against the baseline results obtained by learning each task independently. We also compare with a recently proposed state-of-the-art approach based on support vector machines, obtaining comparable or better results.

Multi-task learning for pKa prediction

“…The task covariance matrix K t can be restricted to a correlation matrix by enforcing a unit diagonal (proper range of off-diagonal elements is ensured by positive definiteness of K t ) [31,32]. We refer to this method as MTL-COR (correlation matrix).…”

Multi-task learning for subthalamic nucleus identification in deep brain stimulation

“…2009), financial time series (Niu and Wang 2014), among others. Inspired by the philosophy behind the multi-task learning framework originated in Machine learning (Caruana 1997;Bakker and Heskes 2003;Bonilla et al 2007;Skolidis and Sanguinetti 2011), we propose in this paper a patient-dependent classification system for basal ganglia identification. The idea behind multi-task learning is that by learning simultaneously different but related tasks, it is possible to increase the performance of a learning algorithm (Argyriou et al 2008).…”

“…We also use the ICM covariance in a multi-task Gaussian process classifier as introduced in Skolidis and Sanguinetti (2011), and refer to this method as MI. 9 In Skolidis and Sanguinetti (2011), the authors use a probit model for relating the observed data y, with the un-observed variables f .…”

“…9 In Skolidis and Sanguinetti (2011), the authors use a probit model for relating the observed data y, with the un-observed variables f . Computing the posterior distribution p(f |y) can not be accomplished in closed form, and the authors use ExpectationPropagation (EP) for computing an approximated posterior.…”

Machine learning in complex networks: modeling, analysis, and applications

“…According to Ref. , graph-based SRC is a within-network classification task and it can be categorized into two main groups: collective inference Macskassy and Provost, 2007;Skolidis and Sanguinetti, 2011;Zhang and Mao, 2008;Zhang et al, 2011Zhang et al, , 2006Zhu et al, 2007) and graph-based semisupervised learning (Chapelle et al, 2006;Silva and Zhao, 2012a,b,c;Zhu, 2005a). In both cases, the labels are propagated from pre-labeled vertices to unlabeled vertices considering the local relationships or certain smoothness criteria.…”