Global Line Search Algorithm Hybridized with Quadratic Interpolation and Its Extension to Separable Functions

Abstract

We propose a novel hybrid algorithm "Brent-STEP" for univariate global function minimization, based on the global line search method STEP and accelerated by Brent's method, a local optimizer that combines quadratic interpolation and golden section steps. We analyze the performance of the hybrid algorithm on various one-dimensional functions and experimentally demonstrate a significant improvement relative to its constituent algorithms in most cases. We then generalize the algorithm to multivariate functions, adopting the recently proposed [8] scheme to interleave evaluations across dimensions to achieve smoother and more efficient convergence. We experimentally demonstrate the highly competitive performance of the proposed multivariate algorithm on separable functions of the BBOB benchmark. The combination of good performance and smooth convergence on separable functions makes the algorithm an interesting candidate for inclusion in algorithmic portfolios or hybrid algorithms that aim to provide good performance on a wide range of problems.

Dimension Selection in Axis-Parallel Brent-STEP Method for Black-Box Optimization of Separable Continuous Functions

“…The multivariate Brent-STEP method for separable functions [2] uses the round-robin (RR) strategy to choose the dimension for the next iteration. We denote this algorithm as BSrr.…”

“…Which solver is a better choice depends on the particular univariate function. The dilemma was recently solved to a great extent [2] by creating a hybrid of the Brent local search method [4] and a global search method called STEP [8]. In most cases, this hybrid called Brent-STEP takes the best of both worlds: it converges quickly on unimodal functions, and still finds the optimum of multimodal functions.…”

“…This general principle was recently proposed for STEP algorithm [9]. The same general principle, albeit with a different implementation details, was used in [2] to improve the performance of the mul-tivariate Brent-STEP method. The dimensions were chosen uniformly in a round-robin (RR) fashion.…”

“…A recently proposed hybrid of these two algorithms, Brent-STEP [2], searches through the already sampled points (interval boundaries) for such triples of consecutive points which bracket a local optimum. To each such triple, it fits a parabola in Brent's way and estimates its optimum.…”

“…Both methods had in common, that until the last dimension started to be optimized, the algorithms showed virtually no progress at all. The Brent-STEP method and its multidimensional generalization [2] mitigated both these issues in the same time: the method works for unimodal functions quickly, for multimodal functions reliably, and shows a progress much sooner during the optimization.…”