Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

Abstract

Selecting the right tuning parameters for algorithms is a pravelent problem in machine learning that can significantly affect the performance of algorithms. Data-efficient optimization algorithms, such as Bayesian optimization, have been used to automate this process. During experiments on real-world systems such as robotic platforms these methods can evaluate unsafe parameters that lead to safety-critical system failures and can destroy the system. Recently, a safe Bayesian optimization algorithm, called SafeOpt, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in practice, since they are often opposing objectives. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle.

Robot Program Parameter Inference via Differentiable Shadow Program Inversion

“…1) Robot skill parameter inference: Several approaches for the automatic optimization of robot skill parameters have been proposed, which rely on gradient-free optimization techniques such as evolutionary algorithms [7], [8], [9] or Bayesian optimization [10], [11], [12] due to the non-differentiability of most skill libraries and frameworks. Gradient-free approaches require frequent execution of the skills during optimization, which is a time-consuming process if done on real robot systems, has to be repeated whenever the task objectives change and often require good initial parameterizations.…”

Uniform Error and Posterior Variance Bounds for Gaussian Process Regression with Application to Safe Control

“…Due to these properties, GPs gained increasing attention in the field of reinforcement learning and system identification. Especially, when safety guarantees are necessary, GPs are favored in reinforcement learning (Berkenkamp et al, 2016a(Berkenkamp et al, ,c, 2017Koller et al, 2018) as well as control (Berkenkamp and Schoellig, 2015;Umlauft et al, 2017;Beckers and Hirche, 2018;Lederer et al, 2020;Umlauft et al, 2018;Helwa et al, 2019). These approaches heavily rely on error bounds of GP regression and are therefore limited by the strict assumptions made in previous works on GP uniform error bounds (Srinivas et al, 2012;Chowdhury and Gopalan, 2017).…”

Cascade Control: Data-Driven Tuning Approach Based on Bayesian Optimization

“…In this work, we restrict the range of the optimization variables to a limited set where the system is stable and focus on overshoot and set point tracking errors. Bayesian optimization in controller tuning where stability is guaranteed through safe exploration has been proposed in (Berkenkamp et al, 2016b), and applied for robotic applications (Berkenkamp et al, 2016a), and in process systems (Khosravi et al, 2019a,b). The proposed Bayesian optimization tuning ensures a compromise between the need of extensive number of trials for finding the optimal gains (according to a specified performance criterion), and a single trial, as resulting from standard methods, where a sub-optimal gain with respect to the performance of the system is found, but stability is ensured for a wide range of operation.…”