Safety-Critical Control using Optimal-decay Control Barrier Function with Guaranteed Point-wise Feasibility

Zeng, Jun; Zhang, Bike; Li, Zhongyu; Sreenath, Koushil

doi:10.23919/acc50511.2021.9482626

“…ω = [ω 1 , ..., ω N −1 ] T represents the relaxation variables of decay rate α DCBF for DCBF constraints. We also notice that ρ in (10) shall be chosen as a relatively large scalar such that the DCBF constraints wouldn't be over-relaxed [73]. Moreover, δ represents the slack variable for terminal constraint on desired final state [x g f , y g f , x φ,f ] T which is minimized with a quadratic term in the cost function.…”

Section: A Safety-critical Navigation Frameworkmentioning

confidence: 99%

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Li¹,

Zeng²,

Thirugnanam³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Bridging model-based safety and model-free reinforcement learning (RL) for dynamic robots is appealing since model-based methods are able to provide formal safety guarantees, while RL-based methods are able to exploit the robot agility by learning from the full-order system dynamics. However, current approaches to tackle this problem are mostly restricted to simple systems. In this paper, we propose a new method to combine model-based safety with model-free reinforcement learning by explicitly finding a low-dimensional model of the system controlled by a RL policy and applying stability and safety guarantees on that simple model. We use a complex bipedal robot Cassie, which is a high dimensional nonlinear system with hybrid dynamics and underactuation, and its RL-based walking controller as an example. We show that a low-dimensional dynamical model is sufficient to capture the dynamics of the closed-loop system. We demonstrate that this model is linear, asymptotically stable, and is decoupled across control input in all dimensions. We further exemplify that such linearity exists even when using different RL control policies. Such results point out an interesting direction to understand the relationship between RL and optimal control: whether RL tends to linearize the nonlinear system during training in some cases. Furthermore, we illustrate that the found linear model is able to provide guarantees by safety-critical optimal control framework, e.g., Model Predictive Control with Control Barrier Functions, on an example of autonomous navigation using Cassie while taking advantage of the agility provided by the RL-based controller.

show abstract

“…ω = [ω 1 , ..., ω N −1 ] T represents the relaxation variables of decay rate α DCBF for DCBF constraints. We also notice that ρ in (10) shall be chosen as a relatively large scalar such that the DCBF constraints wouldn't be over-relaxed [73]. Moreover, δ represents the slack variable for terminal constraint on desired final state [x g f , y g f , x ϕ,f ] T which is minimized with a quadratic term in the cost function.…”

Section: A Safety-critical Navigation Frameworkmentioning

confidence: 99%

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Li¹,

Zeng²,

Thirugnanam³

et al. 2022

Robotics: Science and Systems XVIII

Self Cite

4

0

View full text Add to dashboard Cite

Bridging model-based safety and model-free reinforcement learning (RL) for dynamic robots is appealing since model-based methods are able to provide formal safety guarantees, while RL-based methods are able to exploit the robot agility by learning from the full-order system dynamics. However, current approaches to tackle this problem are mostly restricted to simple systems. In this paper, we propose a new method to combine model-based safety with model-free reinforcement learning by explicitly finding a low-dimensional model of the system controlled by a RL policy and applying stability and safety guarantees on that simple model. We use a complex bipedal robot Cassie, which is a high dimensional nonlinear system with hybrid dynamics and underactuation, and its RL-based walking controller as an example. We show that a low-dimensional dynamical model is sufficient to capture the dynamics of the closed-loop system. We demonstrate that this model is linear, asymptotically stable, and is decoupled across control input in all dimensions. We further exemplify that such linearity exists even when using different RL control policies. Such results point out an interesting direction to understand the relationship between RL and optimal control: whether RL tends to linearize the nonlinear system during training in some cases. Furthermore, we illustrate that the found linear model is able to provide guarantees by safety-critical optimal control framework, e.g., Model Predictive Control with Control Barrier Functions, on an example of autonomous navigation using Cassie while taking advantage of the agility provided by the RL-based controller.

show abstract

“…This approach solves the problem specifically for this system but not for general nonlinear systems. Recently in [19], the decay rates of CBF constraints are relaxed with optimization variables, which generally resolves the conflict between the CBF constraint and the input constraint and guarantees point-wise feasibility.…”

Section: B Related Workmentioning

confidence: 99%

Enhancing Feasibility and Safety of Nonlinear Model Predictive Control with Discrete-Time Control Barrier Functions

Zeng¹,

Li²,

Sreenath³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Safety is one of the fundamental problems in robotics. Recently, one-step or multi-step optimal control problems for discrete-time nonlinear dynamical system are formulated to offer tracking stability using control Lyapunov functions (CLFs) while subject to input constraints as well as safety-critical constraints using control barrier functions (CBFs). The limitations of these existing approaches are mainly about feasibility and safety. In the existing approaches, the optimization feasibility and the system safety cannot be enhanced at the same time theoretically. In this paper, we propose two formulations that unifies CLFs and CBFs under the framework of nonlinear model predictive control (NMPC). In the proposed formulations, safety criteria is commonly formulated as CBF constraints and stability performance is ensured with either a terminal cost function or CLF constraints. Relaxing variables are introduced on the CBF constraints to resolve the tradeoff between feasibility and safety so that they can be enhanced at the same. The advantages about feasibility and safety of proposed formulations compared with existing methods are analyzed theoretically and validated with numerical results.

show abstract

Safety-Critical Control using Optimal-decay Control Barrier Function with Guaranteed Point-wise Feasibility

Cited by 37 publications

References 23 publications

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Enhancing Feasibility and Safety of Nonlinear Model Predictive Control with Discrete-Time Control Barrier Functions

Contact Info

Product

Resources

About