Optimal Bayesian Transfer Learning

Abstract

Transfer learning has recently attracted significant research attention, as it simultaneously learns from different source domains, which have plenty of labeled data, and transfers the relevant knowledge to the target domain with limited labeled data to improve the prediction performance. We propose a Bayesian transfer learning framework, in the homogeneous transfer learning scenario, where the source and target domains are related through the joint prior density of the model parameters. The modeling of joint prior densities enables better understanding of the "transferability" between domains. We define a joint Wishart distribution for the precision matrices of the Gaussian feature-label distributions in the source and target domains to act like a bridge that transfers the useful information of the source domain to help classification in the target domain by improving the target posteriors. Using several theorems in multivariate statistics, the posteriors and posterior predictive densities are derived in closed forms with hypergeometric functions of matrix argument, leading to our novel closed-form and fast Optimal Bayesian Transfer Learning (OBTL) classifier. Experimental results on both synthetic and real-world benchmark data confirm the superb performance of the OBTL compared to the other state-of-the-art transfer learning and domain adaptation methods.

First-Arrival Picking for Microseismic Monitoring Based on Deep Learning

“…Ghahramani has pointed out that Bayesian optimization is one of the most advanced and promising technologies in the artificial intelligence field [53]. Therefore, the following-up research will further study the fast Bayesian optimization scheme for a deep neural network [54], optimize the model, and improve its accuracy [55,56].…”

Deep visual unsupervised domain adaptation for classification tasks: a survey

“…Gholami et al [108] proposed graphical probabilistic unsupervised domain adaptation (PUnDA) model, which learns the classifier in a common space by using the MMD metric with utilising a graphical framework. Karbalayghareh et al [109] presented optimal Bayesian transfer learning (OBTL) model, which combines graph model concept with a Bayesian method for domain adaptation. Perrone et al [110] introduced adaptive Bayesian linear regression (ABLR) model for multi‐task applications, which is a graph‐based method for Bayesian optimisation.…”

Autonomous efficient experiment design for materials discovery with Bayesian model averaging

“…The so-called Bayesian Optimization (BO) [12] in the literature corresponds to these cases, where the prior model is sequentially updated after each experiment. Bayesian parametric and nonparametric models are widely used in other fields such as bioinformatics [13][14][15][16][17][18]. When prior knowledge about the form of the objective function exists and/or many observations of the objective values at different parts of the input space are available, one can use a parametric model as a surrogate model.…”