Constrained Bayesian Optimization with Particle Swarms for Safe Adaptive Controller Tuning

Duivenvoorden, Rikky R. P. R.; Berkenkamp, Felix; Carion, Nicolas; Krause, Andreas; Schoellig, Angela P.

doi:10.1016/j.ifacol.2017.08.1991

Cited by 38 publications

(33 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To do this in a tractable manner, we focus on finite sets A in the following. However, heuristic extensions to continuous domains exist (Duivenvoorden et al, 2017).…”

Section: Safeopt-mc (Multiple Constraints)mentioning

confidence: 99%

Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

2021

Self Cite

View full text Add to dashboard Cite

Selecting the right tuning parameters for algorithms is a pravelent problem in machine learning that can significantly affect the performance of algorithms. Data-efficient optimization algorithms, such as Bayesian optimization, have been used to automate this process. During experiments on real-world systems such as robotic platforms these methods can evaluate unsafe parameters that lead to safety-critical system failures and can destroy the system. Recently, a safe Bayesian optimization algorithm, called SafeOpt, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in practice, since they are often opposing objectives. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle.

show abstract

“…To do this in a tractable manner, we focus on finite sets A in the following. However, heuristic extensions to continuous domains exist (Duivenvoorden et al, 2017).…”

Section: Safeopt-mc (Multiple Constraints)mentioning

confidence: 99%

Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hence, we compare our method to SafeOptSwarm [32] and run only 50 iterations for each algorithm in three independent runs. We attempt a global search after 20 iterations for GoSafeOpt.…”

Section: Hardware Resultsmentioning

confidence: 99%

“…Safe exploration High-dimensional state space Global exploration Sample efficient SafeOpt [24] SafeOptSwarm [32] SafeLineBO [31] GoSafe [29] GoSafeOpt (ours) and provide safety and, optimality guarantees for it. Finally, we demonstrate on software and hardware the superiority of GoSafeOpt over existing methods for safe policy search for high-dimensional problem in Section 5.…”

Section: Contributionsmentioning

confidence: 99%

GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems

Sukhija¹,

Turchetta²,

Lindner³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage. Most existing learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. A notable exception is the GoSafe algorithm, which, unfortunately, cannot handle high-dimensional systems and hence cannot be applied to most real-world dynamical systems. This work proposes GoSafeOpt as the first algorithm that can safely discover globally optimal policies for complex systems while giving safety and optimality guarantees. Our experiments on a robot arm that would be prohibitive for GoSafe demonstrate that GoSafeOpt safely finds remarkably better policies than competing safe learning methods for high-dimensional domains.

show abstract

“…It has been used to safely tune a quadrotor controller for position tracking [6], [24]. In [7], it has been integrated with particle swarm optimization (PSO) to learn high-dimensional controllers. Unfortunately, SAFEOPT may not be sample-efficient due to its exploration strategy.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, the recursive computation of S o t is expensive and not well suited to the fast responses required by adaptive control. Similarly to [7], here we rely on particle swarm optimization (PSO) [34] to solve this optimization problem, which checks that the particles belong to the one-step optimistic safe set as the optimization progresses and avoids computing it explicitly. We initialize m particles positioned uniformly at random within the discretized pessimistic safe set with grid resolution ∆x, velocity ∆x with random sign (L. 2 of Algorithm 2) and fitness equal to the lower bound of the objective l f t (•).…”

Section: Goose For Adaptive Controlmentioning

confidence: 99%

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

König

Turchetta

Lygeros

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Adaptive control approaches yield highperformance controllers when a precise system model or suitable parametrizations of the controller are available. Existing data-driven approaches for adaptive control mostly augment standard model-based methods with additional information about uncertainties in the dynamics or about disturbances. In this work, we propose a purely data-driven, model-free approach for adaptive control. Tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. Thus, our approach builds on GOOSE, an algorithm for safe and sample-efficient Bayesian optimization. We introduce several computational and algorithmic modifications in GOOSE that enable its practical use on a rotational motion system. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation. We further demonstrate the proposed adaptive control approach experimentally on a rotational motion system.

show abstract

Constrained Bayesian Optimization with Particle Swarms for Safe Adaptive Controller Tuning

Cited by 38 publications

References 6 publications

Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

Contact Info

Product

Resources

About