Towards dynamic algorithm selection for numerical black-box optimization

Abstract

One of the most challenging problems in evolutionary computation is to select from its family of diverse solvers one that performs well on a given problem. is algorithm selection problem is complicated by the fact that di erent phases of the optimization process require di erent search behavior. While this can partly be controlled by the algorithm itself, there exist large di erences between algorithm performance. It can therefore be bene cial to swap the con guration or even the entire algorithm during the run. Long deemed impractical, recent advances in Machine Learning and in exploratory landscape analysis give hope that this dynamic algorithm con guration (dynAC) can eventually be solved by automatically trained con guration schedules. With this work we aim at promoting research on dynAC, by introducing a simpler variant that focuses only on switching between di erent algorithms, not congurations. Using the rich data from the Black Box Optimization Benchmark (BBOB) platform, we show that even single-switch dynamic Algorithm selection (dynAS) can potentially result in signicant performance gains. We also discuss key challenges in dynAS, and argue that the BBOB-framework can become a useful tool in overcoming these. CCS CONCEPTS• eory of computation → Bio-inspired optimization; Online algorithms;arXiv:2006.06586v1 [cs.NE]

Leveraging benchmarking data for informed one-shot dynamic algorithm selection

“…We have investigated in this work possibilities to leverage existing benchmark data to derive switch-once dynamic algorithm selection policies. While for some cases the "theoretical" approach suggested in [4] could indeed predict combinations that outperformed the best static solver, the results are less positive for others. One obstacle that hinders an accurate performance prediction are local optima: when the rst algorithm is very good at converging to a local optimum, it is likely to be chosen as 1 .…”

“…Following the approach suggested in [4] we compute a "theoretical" ERT value for all combinations ( 1 , 2 , q B ), where 1 is the rst algorithm, 2 the second, and q B the target value at which we switch from 1 to 2 . To this end, we simply compute ERT( 1 , %, q B ) + ERT( 2 , %, q 5 ) ERT( 2 , %, q B ), where all these ERT values are based on the performance recodes provided in [7].…”

“…On three functions, none of the 100 tested single-switch algorithm combinations was able to outperform the best static solver. The prediction quality of the approach suggested in [4] varies a lot between the dierent functions. While for LeadingOnes, for example, the performance predictions are rather accurate, large discrepancies between predicted and actual performance can be observed for more complex function.…”

Explainable Landscape-Aware Optimization Performance Prediction

“…To solve them, the most appropriate optimization algorithm should be selected and its hyper-parameters should be set. This is a well-known problem of algorithm selection (AS) [1], [2], [3] and algorithm configuration (AC) [4], respectively. The prerequisite to achieve this is performing automated algorithm performance prediction.…”

Leveraging Benchmarking Data for Informed One-Shot Dynamic Algorithm Selection

“…The dynAS is expected to unlock the potential benefit from switching among different algorithms online. Related work has been performed on black-box optimization for numeric optimization [25]. Based on the rich BBOB data set [14], [25] investigates the potential improvement that can be achieved from switching between using solvers.…”