Learning control lyapunov functions from counterexamples and demonstrations

Abstract

We present a technique for learning control Lyapunov-like functions, which are used in turn to synthesize controllers for nonlinear dynamical systems that can stabilize the system, or satisfy specifications such as remaining inside a safe set, or eventually reaching a target set while remaining inside a safe set. The learning framework uses a demonstrator that implements a black-box, untrusted strategy presumed to solve the problem of interest, a learner that poses finitely many queries to the demonstrator to infer a candidate function, and a verifier that checks whether the current candidate is a valid control Lyapunov function. The overall learning framework is iterative, eliminating a set of candidates on each iteration using the counterexamples discovered by the verifier and the demonstrations over these counterexamples. We prove its convergence using ellipsoidal approximation techniques from convex optimization. We also implement this scheme using nonlinear MPC controllers to serve as demonstrators for a set of state and trajectory stabilization problems for nonlinear dynamical systems. We show how the verifier can be constructed efficiently using convex relaxations of the verification problem for polynomial systems to semi-definite programming (SDP) problem instances. Our approach is able to synthesize relatively simple polynomial control Lyapunov functions, and in that process replace the MPC using a guaranteed and computationally less expensive controller.

Phase portraits, Lyapunov functions, and projective geometry

“…(Cf. [12,14,17,18]; also see [4] and [16] for relevant background.) Nevertheless, we feel that the purely computational approach described here (and used only in a specific example) has its merits, and at the end of this paper we will make some didactical remarks concerning the relation between concrete calculations and abstract reasoning and also the relation between specific examples and general theories.…”

Formal Policy Learning from Demonstrations for Reachability Properties

“…In this section, we review two main sets of solutions: (a) A verifier that attempt to approximately solve the verification problems using decision procedures, as described in our earlier work [25]. Such a verifier concludes that the system satisfies the property or is likely buggy.…”

Automated and Formal Synthesis of Neural Barrier Certificates for Dynamical Models

“…Whilst Lyapunov theory is classically approached either analytically (explicit synthesis) or numerically (with unsound techniques), an approach that is relevant for the results of this work looks at automated and sound Lyapunov function synthesis: in [27] Lyapunov functions are soundly found within parametric templates, by constructing a system of linear inequality constraints over unknown coefficients. [23,24,25] employ a counterexample-based approach to synthesise control Lyapunov functions, which inspires this work, using a combination of SMT solvers and convex optimisation engines: however unlike this work, SMT solvers are never used for verification, which is instead handled by solving optimisation problems that are numerically unsound. As argued above, let us emphasise again that the BC synthesis problem, as studied in this work, cannot in general be reduced to a problem of Lyapunov stability analysis, and is indeed more general.…”