Sablas: Learning Safe Control for Black-Box Dynamical Systems

Qin, Zhongjun; Sun, Dawei; Fan, Chuchu

doi:10.1109/lra.2022.3142743

Cited by 18 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Classical control: We implement a model predictive control baseline via the CasADi solver [30]. • Safe control: Constrained policy optimization (CPO) [31] and a model-based safe control approach SABLAS [32]. The motivation for our choice of a diverse set of baseline algorithms stems from the perspective of a user trying to solve the problem of safe navigation, who would likely select one of these these established approaches found in the robotics literature.…”

Section: B Comparison With Baseline Methodsmentioning

confidence: 99%

ConBaT: Control Barrier Transformer for Safe Policy Learning

Meng¹,

Vemprala²,

Bonatti³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large-scale self-supervised models have recently revolutionized our ability to perform a variety of tasks within the vision and language domains. However, using such models for autonomous systems is challenging because of safety requirements: besides executing correct actions, an autonomous agent must also avoid the high cost and potentially fatal critical mistakes. Traditionally, self-supervised training mainly focuses on imitating previously observed behaviors, and the training demonstrations carry no notion of which behaviors should be explicitly avoided. In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion. ConBaT is inspired by the concept of control barrier functions in control theory and uses a causal transformer that learns to predict safe robot actions autoregressively using a critic that requires minimal safety data labeling. During deployment, we employ a lightweight online optimization to find actions that ensure future states lie within the learned safe set. We apply our approach to different simulated control tasks and show that our method results in safer control policies compared to other classical and learning-based methods such as imitation learning, reinforcement learning, and model predictive control.

show abstract

Section: B Comparison With Baseline Methodsmentioning

confidence: 99%

ConBaT: Control Barrier Transformer for Safe Policy Learning

Meng¹,

Vemprala²,

Bonatti³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In this case, L V is used to self-supervise both the certificate V and control policy π. The first example of simultaneous certificate/policy learning appeared in [7] for a Lyapunov certificate; later works learn barrier functions [10] and contraction metrics [13] alongside control policies as well, or extend to the case when dynamics are only partially-known [43].…”

Section: Learning Neural Certificatesmentioning

confidence: 99%

“…Other authors, notably Tsukamoto, Chung, and Slotine [34], [51] have extended the neural certificate approach to contraction metrics (see [5] for an overview of these approaches). In addition to different types of certificate, later works have expanded this framework to more challenging problems in control, such as multi-agent control in [10], black-box dynamical models in [43], the RL context in [8], and the robust case in [9]. Many of these later works are particularly notable for providing theoretical and algorithmic contributions to address difficulties that arise in practice (such as control from observations), which we discuss next in Section V.…”

Section: History Of Certificate Learningmentioning

confidence: 99%

“…1) Model-free certificate learning and theoretical connections to RL: Although works like [43] and [8] deploy certificate learning in the black-box dynamics model and unknown-model settings, respectively, deeper connections between certificate learning and reinforcement learning remain unexplored. For example, [62] shows that it is possible to construct a reward function for specific RL problems so that the corresponding value function is also a Lyapunov function, but it is not well understood whether this is possible more generally.…”

Section: B Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods

Dawson¹,

Gao²,

Fan³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies -these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this paper, we provide a comprehensive survey of this rapidly developing field of certificate learning. We hope that this paper will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control.

show abstract

“…Then [22,23] propose a contraction-metric-based control framework, which extends neural networks to certificate learning for contraction metrics. Moreover, based on the framework proposed by Tsukamoto et al, [24,25,26,8] are used to address higher dimensional control problems.…”

Section: Related Workmentioning

confidence: 99%

QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking

Wang¹,

O’Keeffe²,

Qian³

et al. 2022

Preprint

View full text Add to dashboard Cite

Accuracy and stability are common requirements for Quadrotor trajectory tracking systems. Designing an accurate and stable tracking controller remains challenging, particularly in unknown and dynamic environments with complex aerodynamic disturbances. We propose a Quantile-approximation-based Distributional-reinforced Uncertainty Estimator (QuaDUE) to accurately identify the effects of aerodynamic disturbances, i.e., the uncertainties between the true and estimated Control Contraction Metrics (CCMs). Taking inspiration from contraction theory and integrating the QuaDUE for uncertainties, our novel CCMbased trajectory tracking framework tracks any feasible reference trajectory precisely whilst guaranteeing exponential convergence. More importantly, the convergence and training acceleration of the distributional RL are guaranteed and analyzed, respectively, from theoretical perspectives. We also demonstrate our system under unknown and diverse aerodynamic forces. Under large aerodynamic forces (>2 m/s 2 ), compared with the classic data-driven approach, our QuaDUE-CCM achieves at least a 56.6% improvement in tracking error. Compared with QuaDRED-MPC, a distributional RL-based approach, QuaDUE-CCM achieves at least a 3 times improvement in contraction rate.

show abstract

Sablas: Learning Safe Control for Black-Box Dynamical Systems

Cited by 18 publications

References 28 publications

ConBaT: Control Barrier Transformer for Safe Policy Learning

ConBaT: Control Barrier Transformer for Safe Policy Learning

Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods

QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking

Contact Info

Product

Resources

About