Bayesian Optimization for Policy Search in High-Dimensional Systems via Automatic Domain Selection

Fröhlich, Lukas P.; Klenske, Edgar D.; Daniel, Christian; Zeilinger, Melanie N.

doi:10.48550/arxiv.2001.07394

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These functions are also known as objectives with 'active subspaces' [14] or 'multiridge' [25,55]. They are frequently encountered in applications, typically when tuning (over)parametrized models and processes, such as in hyper-parameter optimization for neural networks [3], heuristic algorithms for combinatorial optimization problems [34], complex engineering and physical simulation problems [14] as in climate modelling [37], and policy search and dynamical system control [26,61].…”

Section: Introductionmentioning

confidence: 99%

Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings

2022

View full text Add to dashboard Cite

We consider the bound-constrained global optimization of functions with low effective dimensionality, that are constant along an (unknown) linear subspace and only vary over the effective (complement) subspace. We aim to implicitly explore the intrinsic low dimensionality of the constrained landscape using feasible random embeddings, in order to understand and improve the scalability of algorithms for the global optimization of these special-structure problems. A reduced subproblem formulation is investigated that solves the original problem over a random low-dimensional subspace subject to affine constraints, so as to preserve feasibility with respect to the given domain. Under reasonable assumptions, we show that the probability that the reduced problem is successful in solving the original, full-dimensional problem is positive. Furthermore, in the case when the objective’s effective subspace is aligned with the coordinate axes, we provide an asymptotic bound on this success probability that captures its polynomial dependence on the effective and, surprisingly, ambient dimensions. We then propose X-REGO, a generic algorithmic framework that uses multiple random embeddings, solving the above reduced problem repeatedly, approximately and possibly, adaptively. Using the success probability of the reduced subproblems, we prove that X-REGO converges globally, with probability one, and linearly in the number of embeddings, to an $$\epsilon $$ ϵ -neighbourhood of a constrained global minimizer. Our numerical experiments on special structure functions illustrate our theoretical findings and the improved scalability of X-REGO variants when coupled with state-of-the-art global—and even local—optimization solvers for the subproblems.

show abstract

Section: Introductionmentioning

confidence: 99%

Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings

2022

View full text Add to dashboard Cite

show abstract

“…The task was to stabilize a Furuta pendulum (Furuta et al, 1992) for 5 s around the upper equilibrium position using a linear state-feedback controller. We applied BO to tune the four feedback gains of this controller (Fröhlich et al, 2020). To assess the performance of a given controller, we employed a logarithmic quadratic cost function (Bansal et al, 2017).…”

Section: Methodsmentioning

confidence: 99%

Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

Volpp¹,

Fröhlich²,

Fischer³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Transferring knowledge across tasks to improve data-efficiency is one of the open key challenges in the field of global black-box optimization. Readily available algorithms are typically designed to be universal optimizers and, therefore, often suboptimal for specific tasks. We propose a novel transfer learning method to obtain customized optimizers within the well-established framework of Bayesian optimization, allowing our algorithm to utilize the proven generalization capabilities of Gaussian processes. Using reinforcement learning to meta-train an acquisition function (AF) on a set of related tasks, the proposed method learns to extract implicit structural information and to exploit it for improved data-efficiency. We present experiments on a simulation-to-real transfer task as well as on several synthetic functions and on two hyperparameter search problems. The results show that our algorithm (1) automatically identifies structural properties of objective functions from available source tasks or simulations, (2) performs favourably in settings with both scarse and abundant source data, and (3) falls back to the performance level of general AFs if no particular structure is present.

show abstract

Bayesian Optimization for Policy Search in High-Dimensional Systems via Automatic Domain Selection

Cited by 2 publications

References 0 publications

Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings

Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings

Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

Contact Info

Product

Resources

About