Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude

Abstract

Building on the recently developed theory of magnitude, we introduce the optimization algorithm EXPLO2 and carefully benchmark it. EXPLO2 advances the state of the art for optimizing highdimensional (D ⪆ 40) multimodal functions that are expensive to compute and for which derivatives are not available, such as arise in hyperparameter optimization or via simulations. Example 1. Let {x j } 3 j=1 ⊂ R 2 have pairwise distances d jk ∶= d(x j , x k ) given by d 12 = d 13 = 1 = d 21 = d 31 and d 23 = δ = d 32 with δ < 2. It turns out that w 1 = e (δ+2)t − 2e (δ+1)t + e 2t e (δ+2)t − 2e δt + e 2t ;

Benchmarking an algorithm for expensive high-dimensional objectives on the bbob and bbob-largescale testbeds

“…In high dimensions it is appropriate to compare EXPLO2 to large-scale variants of CMA-ES [23,24], and our ad hoc experiments in this regard yielded good results. 6 Also, in an experiment shown in an appendix of [13], we found that EXPLO2 outperformed differential evolution and had performance almost indistinguishable from CMA-ES on the mixed-integer version of 𝑓 15 [21] in dimension 𝐷 = 20 (the only function/dimension pair we tried among the mixed-integer suite [21]).…”

“…We ran EXPLO2 with default settings as described in [13] and with 𝑛 ∥ = 32 parallel workers on the entire bbob suite for 25 × 𝐷 evaluations according to [12]. We also ran EXPLO2 in the same way on the multimodal functions 𝑓 15 , .…”

“…As [13] details, a representative set of algorithms to benchmark against for low-dimensional, expensive, multimodal functions is NEWUOA, MCS, GLOBAL, SMAC-BBOB, and *-CMA-ES with * ∈ {lmm, DTS, lq}. Of these, only NEWUOA is well-suited to problems with hundreds of dimensions.…”

“…The paper [13] introduces an optimization algorithm designed to handle intrinsically high-dimensional (𝐷 ⪆ 40, and with all dimensions relevant) optimization problems for very expensive (≥ 1 minute/evaluation) multimodal objective functions. This regime includes problems of fundamental interest (e.g., optimizing the results of simulations) that push against fundamental limits of benchmarking tools: for example, high dimensionality and multimodality coupled with low function evaluation budgets preclude achieving global minima as a realistic goal.…”

“…In an appendix of[13], we also produced fixed-budget plots of cumulative best values using the same data (and substituting, e.g. DTS-CMA-ES for lmm-CMA-ES) via the web interface of IOHanalyzer[3] at https://iohanalyzer.liacs.nl/, but these plots produced neither additional nor conflicting insights.…”