MUMBO: MUlti-task Max-value Bayesian Optimization

Abstract

We propose MUMBO, the first high-performing yet computationally efficient acquisition function for multi-task Bayesian optimization. Here, the challenge is to perform efficient optimization by evaluating low-cost functions somehow related to our true target function. This is a broad class of problems including the popular task of multi-fidelity optimization. However, while information-theoretic acquisition functions are known to provide state-of-the-art Bayesian optimization, existing implementations for multi-task scenarios have prohibitive computational requirements. Previous acquisition functions have therefore been suitable only for problems with both low-dimensional parameter spaces and function query costs sufficiently large to overshadow very significant optimization overheads. In this work, we derive a novel multi-task version of entropy search, delivering robust performance with low computational overheads across classic optimization challenges and multi-task hyperparameter tuning. MUMBO is scalable and efficient, allowing multi-task Bayesian optimization to be deployed in problems with rich parameter and fidelity spaces.

Using Distance Correlation for Efficient Bayesian Optimization

“…That search goes under the name of Outputspace Predictive Entropy Search (OPES) (Hoffman and Ghahramani, 2015) or Max-value Entropy Search (MES) (Wang and Jegelka, 2017). And MES has been recently generalized to multi-fidelity BO (Takeno et al, 2020;Moss et al, 2020).…”

“…Second, we assume that all channels of observations are of equal cost. Third, we focus on the use of DC, in contrast to (Takeno et al, 2020;Moss et al, 2020) that solely considered MI as the measure of information gain. Fourth, we incorporate integral observations of a black-box function, which has not been considered before in the literature of multi-fidelity BO.…”

Gaussian Process Molecule Property Prediction with FlowMO

“…Finally, FlowMO will allow us to apply the extended Bayesian optimization methods designed for GP models to further improve efficiency in automatic wet-lab workflows [MacLeod et al, 2020]. Potential extensions include batch [González et al, 2016], multitask [Swersky et al, 2013], multi-fidelity [Moss et al, 2020c] and multi-objective optimization, as well as optimization with controllable experimental noise [Moss et al, 2020b].…”

Evolutionary Multitask Optimization: a Methodological Overview, Challenges and Future Research Directions

“…A significant effort has been conducted by the community for solving a wide variety of continuous, combinatorial, single-objective and multiobjective optimization problems through the perspective of EM [9,10,11,12]. Another research direction for dealing with multitasking in the context of TO is multitask Bayesian optimization [13], which extends Bayesian optimization approaches to multitasking environments [14,15,16,17]. Despite falling out of the focus of this paper due to its non-evolutionary nature, we note that Bayesian solvers, along those within EM, constitute the core of the contributions reported in the field of multitasking, with a significantly higher presence of EM methods.…”