Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods

Abstract

Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies -these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this paper, we provide a comprehensive survey of this rapidly developing field of certificate learning. We hope that this paper will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control.

QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking

“…Then [22,23] propose a contraction-metric-based control framework, which extends neural networks to certificate learning for contraction metrics. Moreover, based on the framework proposed by Tsukamoto et al, [24,25,26,8] are used to address higher dimensional control problems.…”

“…where P W B , V W B and q W B are the position, linear velocity and orientation expressed in the world frame, and ω B is the angular velocity expressed in the body frame (more detailed definitions please see [10]). Then we reformulate Equation 14 in its control-affine form: ẋ = f (x, e f k )+g(x)u+w, Where f : R n → R n and g : R n → R n×m are assumed by the standard as Lipschitz continuous [29,8].…”

“…, where θ nominal−metric is the parameter of the neural network in the contraction metric learning [8] for the nominal function Equation 14with e f = 0, then the truth C m under uncertainty becomes:…”

“…Performance of QuaDUE Training: we have two main training process: firstly, we employ a certified learning controller proposed in [8,31] to learn a Contraction Metric for the nominal model (Equation 14) as shown in Fig. 1.…”

“…The second extends the contraction theory [7] to a control-affine system, called Control Contraction Metric (CCM). This contraction certificate guarantees that the system will exponentially converge to track any feasible reference trajectory [8]. In applications such as quadrotor tracking, we are concerned with the whole tracking process rather than just stabilizing to a fixed point.…”