Expected hypervolume improvement for simultaneous multi-objective and multi-fidelity optimization

Abstract

Bayesian optimization has proven to be an efficient method to optimize expensiveto-evaluate systems. However, depending on the cost of single observations, multi-dimensional optimizations of one or more objectives may still be prohibitively expensive. Multi-fidelity optimization remedies this issue by including multiple, cheaper information sources such as low-resolution approximations in numerical simulations. Acquisition functions for multi-fidelity optimization are typically based on exploration-heavy algorithms that are difficult to combine with optimization towards multiple objectives.Here we show that the expected hypervolume improvement policy can act in many situations as a suitable substitute. We incorporate the evaluation cost either via a two-step evaluation or within a single acquisition function with an additional fidelity-related objective. This permits simultaneous multi-objective and multifidelity optimization, which allows to accurately establish the Pareto set and front at fractional cost. Benchmarks show a cost reduction of an order of magnitude or more. Our method thus allows for Pareto optimization of extremely expansive black-box functions. The presented methods are simple and straightforward to implement in existing, optimized Bayesian optimization frameworks and can immediately be extended to batch optimization. The techniques can also be used to combine different continuous and/or discrete fidelity dimensions, which makes them particularly relevant for simulation problems in plasma physics, fluid dynamics and many other branches of scientific computing.Preprint. Under review.

Multi-BOWS: multi-fidelity multi-objective Bayesian optimization with warm starts for nanophotonic structure design

“…[67][68][69] In parallel, multi-objective Bayesian optimization has been developed to optimize multiple objectives simultaneously. [70][71][72] Recent research efforts have sought to combine these two concepts into multi-delity multi-objective Bayesian optimization by the introduction of continuous delity levels as an optimizeable parameter 73,74 or aiming to maximize information gain per unit cost of resources. 25…”

Data-driven science and machine learning methods in laser–plasma physics

“…These multi-information-source methods have the potential to speed up optimization significantly. They can also be combined with multi-objective optimization, as shown by Irshad et al [199] .…”

Multi-fidelity Multi-Objective Optimization of a High-Altitude Propeller

“…Suppose another metamodel is used whose training time for large data is feasible, such as deep neural networks which easily use modern computational architectures such as GPUs. In that case, the metamodel's property of the prediction variance is lost, which is essential for the acquisition functions of multi-fidelity Bayesian Optimization methods [28][29][30]. There are many different classes of models other than GPs that provide a prediction uncertainty for an unknown point.…”

“…The approach is similar to the one proposed by Jeong S. et al [42] for multi-objective EGO, but it further chooses the solver's fidelity by evaluating the multi-fidelity metamodel error prediction. Finally, some information-based MFMO acquisition functions have been proposed [29,30] that are considered less myopic (i.e. not focused on the immediate goal of the next iteration).…”