Multi-fidelity Gaussian Process Bandit Optimisation

Abstract

In many scientific and engineering applications, we are tasked with the maximisation of an expensive to evaluate black box function f . Traditional settings for this problem assume just the availability of this single function. However, in many cases, cheap approximations to f may be obtainable. For example, the expensive real world behaviour of a robot can be approximated by a cheap computer simulation. We can use these approximations to eliminate low function value regions cheaply and use the expensive evaluations of f in a small but promising region and speedily identify the optimum. We formalise this task as a multi-fidelity bandit problem where the target function and its approximations are sampled from a Gaussian process. We develop MF-GP-UCB, a novel method based on upper confidence bound techniques. In our theoretical analysis we demonstrate that it exhibits precisely the above behaviour and achieves better bounds on the regret than strategies which ignore multi-fidelity information. Empirically, MF-GP-UCB outperforms such naive strategies and other multi-fidelity methods on several synthetic and real experiments. KANDASAMY, DASARATHY, OLIVA, SCHNEIDER, PÓCZOS n=300 n=3000 1. We present a formalism for multi-fidelity bandit optimisation using Gaussian process (GP) assumptions on f and its approximations. We develop a novel algorithm, Multi-Fidelity Gaussian Process Upper Confidence Bound (MF-GP-UCB) for this setting.2. Our theoretical analysis proves that MF-GP-UCB explores the space X at lower fidelities and uses the high fidelities in successively smaller regions to converge on the optimum. As lower fidelity queries are cheaper, MF-GP-UCB has better upper bounds on the regret than single fidelity strategies which have to rely on the expensive function to explore the entire space.3. We demonstrate that MF-GP-UCB outperforms single fidelity methods and other alternatives empirically, via a series of synthetic examples, three hyper-parameter tuning tasks and one inference problem in astrophysics. Our matlab implementation and experiments are available at github.com/kirthevasank/mf-gp-ucb. Related WorkSince the seminal work by Robbins [1952], the multi-armed bandit problem has been studied extensively in the K-armed setting. Recently, there has been a surge of interest in the optimism under uncertainty principle for K-armed bandits, typified by upper confidence bound (UCB) methods Cesa-Bianchi, 2012, Auer, 2003]. UCB strategies have also been used in bandit

Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization

“…A GP model with multiple information sources was first considered in [20]. Since then, optimization with multiple information sources has been considered under strict requirements, such as models forming a hierarchy of increasing accuracy and without considering different costs [21], [22]. More recently, [23] used a myopic policy, called the 'knowledge gradient' by [24], in order determine, which parameters to evaluate.…”

A Non-Nested Infilling Strategy for Multifidelity Based Efficient Global Optimization

“…As a final word on computational complexity, we observe that finding the optimum of the merit function calls for multiple estimation of the updated variance b (L) (x|x, l), which in turn calls for the computation of variance reduction terms, such as b (l) (x|x, l), defined by (22). The computationally intensive part of this evaluation is the inversion of the GP matrices in (22). The second one, K (l) , is not depending on the new point x ⇤ such that it can be factorized and inverted once for all during the search of the couple (x ⇤ , l ⇤ ) maximizing the merit function.…”

“…The work in [21] presents a MF optimization framework for a continuous set of variable fidelity models. In [22], the authors propose an alternative optimization strategy using a low-fidelity model to cheaply eliminate regions of poor performance and a high-fidelity model in the promising regions only.…”

Gaussian process bandits with adaptive discretization

“…Krause and Ong (2011) proposed and analyzed the CGP-UCB algorithm for the contextual GP bandits problem, where the mean reward function corresponding to context-action pairs is modeled as a sample from a GP on the context-action product space. Kandasamy et al (2016) considered a multi-fidelity version of the GP bandits problem in which they assumed the availability of a sequence of approximations of the true function f with increasing accuracies which were cheaper to evaluate. They proposed an extension of GP-UCB called the MF-GP-UCB and derived information-type bounds on its cumulative regret.…”