Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

Abstract

Transferring knowledge across tasks to improve data-efficiency is one of the open key challenges in the field of global black-box optimization. Readily available algorithms are typically designed to be universal optimizers and, therefore, often suboptimal for specific tasks. We propose a novel transfer learning method to obtain customized optimizers within the well-established framework of Bayesian optimization, allowing our algorithm to utilize the proven generalization capabilities of Gaussian processes. Using reinforcement learning to meta-train an acquisition function (AF) on a set of related tasks, the proposed method learns to extract implicit structural information and to exploit it for improved data-efficiency. We present experiments on a simulation-to-real transfer task as well as on several synthetic functions and on two hyperparameter search problems. The results show that our algorithm (1) automatically identifies structural properties of objective functions from available source tasks or simulations, (2) performs favourably in settings with both scarse and abundant source data, and (3) falls back to the performance level of general AFs if no particular structure is present.

The Differentiable Cross-Entropy Method

“…We find this especially promising for meta-learning, potentially building on LEO (Rusu et al, 2018). Inspired by DCEM, other more powerful sampling-based optimizers could be made differentiable in the same way, potentially optimizers that leverage gradient-based information in the inner optimization steps (Sekhon & Mebane, 1998;Theodorou et al, 2010;Stulp & Sigaud, 2012;Maheswaranathan et al, 2018) or by also learning the hyper-parameters of structured optimizers (Li & Malik, 2016;Volpp et al, 2019;Chen et al, 2017).…”

Meta-Surrogate Benchmarking for Hyperparameter Optimization

“…A possible extension of PROFET would be to consider multi-fidelity benchmarks (Klein et al, 2017a;Kandasamy et al, 2017;Klein et al, 2017c) where cheap, but approximate fidelities of the objective function are available, e. g. learning curves or dataset subsets. Furthermore, since PROFET also provides gradient information it could serve as a training distribution for learning-to-learn approaches (Chen et al, 2017;Volpp et al, 2019).…”

Equipping data-driven experiment planning for Self-driving Laboratories with semantic memory: case studies of transfer learning in chemical reaction optimization

“…This may be done by i) updating the predictive mean and variance of the surrogate model learned in the target campaign, 42 or ii) by means of an ensemble of acquisition functions. 43,44 Prior work has argued that the latter option better aggregates information from source campaigns with varying objective value ranges 43 (see ESI † Sec. S.2.A and S.2.B for additional details).…”

“…Sets of source measurements are then sampled from these perturbed surfaces. Additional details on the generation (following the procedure proposed by Volpp et al 44 ) of source data are given in ESI † Sec. S. 4.…”