Black-box combinatorial optimization using models with integer-valued minima

Abstract

When a black-box optimization objective can only be evaluated with costly or noisy measurements, most standard optimization algorithms are unsuited to find the optimal solution. Specialized algorithms that deal with exactly this situation make use of surrogate models. These models are usually continuous and smooth, which is beneficial for continuous optimization problems, but not necessarily for combinatorial problems. However, by choosing the basis functions of the surrogate model in a certain way, we show that it can be guaranteed that the optimal solution of the surrogate model is integer. This approach outperforms random search, simulated annealing and a Bayesian optimization algorithm on the problem of finding robust routes for a noise-perturbed traveling salesman benchmark problem, with similar performance as another Bayesian optimization algorithm, and outperforms all compared algorithms on a convex binary optimization problem with a large number of variables.

Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

“…On the one hand, these surrogate models lend themselves well to problems with only continuous variables, but not so much when they include integer variables as well. On the other hand, there have been several recent approaches to develop surrogate models for problems with only discrete variables [1,6,11,27].…”

“…Most of the methods mentioned here suffer from the drawback that the surrogate model grows while the algorithm is running, causing the algorithms to slow down over time. This problem has been addressed and solved for the continuous setting in the DONE algorithm [5] and for the discrete setting in the COMBO [27] and IDONE algorithms [6] by making use of parametric surrogate models that are linear in the parameters. The MiVaBO algorithm [9] is, to the best of our knowledge, the first algorithm that applies this solution to the mixed variable setting.…”

“…• choose them directly according to the data samples in a non-parametric way using kernel basis functions [21,25], • choose them randomly once and then fix them [4,5,9,27], or • choose them according to the variable domains , and then fix them [6].…”

“…Definition 1 (Integer -function). An integer -function is chosen according to (4) with v = 0 and with w and having integer values chosen according to Algorithm 2 from [6]. That means it has one of the following forms: (x , x ) = (x ) = ±( − ), with an element from x and ∈ Z chosen between and (the lower and upper bounds of ), or (x , x ) = (x ) = ±( − −1 − ), for > 1 and ∈ Z chosen between − −1 and − −1 .…”

“…While most attention in the surrogate modelling literature has gone to problems in continuous domains, recently solutions for combinatorial optimisation problems have started to arise [1,2,6,11,27]. Yet a significant number of problems [7] contain a mix of continuous and discrete variables, for example material design [14], optical filter optimisation [30], and automated machine learning [13].…”