CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels

Abstract

Bayesian optimization is a popular method for optimizing expensive black-box functions. The objective functions of hard real world problems are oftentimes characterized by a fluctuated landscape of many local optima. Bayesian optimization risks in overexploiting such traps, remaining with insufficient query budget for exploring the global landscape. We introduce Coordinate Backoff Bayesian optimization (CobBO) to alleviate those challenges. CobBO captures a smooth approximation of the global landscape by interpolating the values of queried points projected to randomly selected promising coordinate subspaces. Thus also a smaller query budget is required for the Gaussian process regressions applied over the lower dimensional subspaces. This approach can be viewed as a variant of coordinate ascent, tailored for Bayesian optimization, using a stopping rule for backing off from a certain subspace and switching to another coordinate subset. Additionally, adaptive trust regions are dynamically formed to expedite the convergence, and stagnant local optima are escaped by switching trust regions. Further smoothness and acceleration are achieved by filtering out clustered queried points. Through comprehensive evaluations over a wide spectrum of benchmarks, CobBO is shown to consistently find comparable or better solutions, with a reduced trial complexity compared to the state-of-the-art methods in both low and high dimensions.

Learning-To-Ensemble by Contextual Rank Aggregation in E-Commerce

“…In particular, Bayesian Optimization uses a Bayesian statistical model, such as Gaussian Process (GP) regression, to model the objective function and relies on an Acquisition Function (AF) to decide the next sample to interact with the system. In many scenarios including ours, GP regression is not enough and needs to be replaced by kernel methods [23], random forest [24], neural network [25], and others [26], which can be summarized as surrogate-based BBO methods [27]. Monte Carlo Tree Search (MCTS) [28] is another popular algorithm for BBO used extensively in adversarial games and robotics planning.…”