Bayesian Optimization for Policy Search in High-Dimensional Systems via Automatic Domain Selection

Fröhlich, Lukas P.; Klenske, Edgar D.; Daniel, Christian; Zeilinger, Melanie N.

doi:10.1109/iros40897.2019.8967736

Cited by 12 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of unconstrained optimal control, Marco et al (104) introduced a parametric cost function l(x, u, θ l ) = x T Q(θ l )x + u T R(θ l )u and optimized the parameters θ l in order to compensate for deviations of the true dynamics f t from linear prediction dynamics f, such that the performance metric in closed loop is improved. In a similar setup, Fröhlich et al (105) investigated learning in higher-dimensional spaces by automatic domain adaptation. For constrained optimal control, the approach of Bansal et al (106) instead uses Bayesian optimization to learn the parameters θ f of a linear prediction model f(x, u, θ f ) = A(θ f )x + B(θ f )u in an MPC, similarly with the goal of improving the closed-loop performance of a specific task for unknown, potentially nonlinear and stochastic true system dynamics f t .…”

Section: Bayesian Optimization For Controller Tuningmentioning

confidence: 99%

Learning-Based Model Predictive Control: Toward Safe Learning in Control

Hewing

Wabersich

Menner

et al. 2020

Annu. Rev. Control Robot. Auton. Syst.

Self Cite

507

269

View full text Add to dashboard Cite

Recent successes in the field of machine learning, as well as the availability of increased sensing and computational capabilities in modern control systems, have led to a growing interest in learning and data-driven control techniques. Model predictive control (MPC), as the prime methodology for constrained control, offers a significant opportunity to exploit the abundance of data in a reliable manner, particularly while taking safety constraints into account. This review aims at summarizing and categorizing previous research on learning-based MPC, i.e., the integration or combination of MPC with learning methods, for which we consider three main categories. Most of the research addresses learning for automatic improvement of the prediction model from recorded data. There is, however, also an increasing interest in techniques to infer the parameterization of the MPC controller, i.e., the cost and constraints, that lead to the best closed-loop performance. Finally, we discuss concepts that leverage MPC to augment learning-based controllers with constraint satisfaction properties.

show abstract

Section: Bayesian Optimization For Controller Tuningmentioning

confidence: 99%

Learning-Based Model Predictive Control: Toward Safe Learning in Control

Hewing

Wabersich

Menner

et al. 2020

Annu. Rev. Control Robot. Auton. Syst.

Self Cite

507

269

View full text Add to dashboard Cite

show abstract

“…These functions are also known as objectives with 'active subspaces' [12] or 'multi-ridge' [23,52]. They are frequently encountered in applications, typically when tuning (over)parametrized models and processes, such as in hyper-parameter optimization for neural networks [3], heuristic algorithms for combinatorial optimization problems [32], complex engineering and physical simulation problems [12] as in climate modelling [35], and policy search and dynamical system control [57,24].…”

Section: Introductionmentioning

confidence: 99%

Constrained global optimization of functions with low effective dimensionality using multiple random embeddings

Cartis¹,

Massart²,

Otemissov³

2020

Preprint

View full text Add to dashboard Cite

We consider the bound-constrained global optimization of functions with low effective dimensionality, that are constant along an (unknown) linear subspace and only vary over the effective (complement) subspace. We aim to implicitly explore the intrinsic low dimensionality of the constrained landscape using feasible random embeddings, in order to understand and improve the scalability of algorithms for the global optimization of these special-structure problems. A reduced subproblem formulation is investigated that solves the original problem over a random low-dimensional subspace subject to affine constraints, so as to preserve feasibility with respect to the given domain. Under reasonable assumptions, we show that the probability that the reduced problem is successful in solving the original, full-dimensional problem is positive. Furthermore, in the case when the objective's effective subspace is aligned with the coordinate axes, we provide an asymptotic bound on this success probability that captures its algebraic dependence on the effective and, surprisingly, ambient dimensions. We then propose X-REGO, a generic algorithmic framework that uses multiple random embeddings, solving the above reduced problem repeatedly, approximately and possibly, adaptively. Using the success probability of the reduced subproblems, we prove that X-REGO converges globally, with probability one, and linearly in the number of embeddings, to an -neighbourhood of a constrained global minimizer. Our numerical experiments on special structure functions illustrate our theoretical findings and the improved scalability of X-REGO variants when coupled with state-of-the-art global -and even localoptimization solvers for the subproblems.

show abstract

“…A few approaches have been proposed in this context to handle input uncertainty. 53,54 Most recently, Fröhlich et al 55 have introduced an acquisition function for GP-based Bayesian optimization for the identification of robust optima. This formulation is analytically intractable and the authors propose two numerical approximation schemes.…”

Section: Background and Related Workmentioning

confidence: 99%