Learning Lyapunov (Potential) Functions from Counterexamples and Demonstrations

Abstract

Abstract-We present a technique for learning control Lyapunov (potential) functions, which are used in turn to synthesize controllers for nonlinear dynamical systems. The learning framework uses a demonstrator that implements a black-box, untrusted strategy presumed to solve the problem of interest, a learner that poses finitely many queries to the demonstrator to infer a candidate function and a verifier that checks whether the current candidate is a valid control Lyapunov function. The overall learning framework is iterative, eliminating a set of candidates on each iteration using the counterexamples discovered by the verifier and the demonstrations over these counterexamples. We prove its convergence using ellipsoidal approximation techniques from convex optimization. We also implement this scheme using nonlinear MPC controllers to serve as demonstrators for a set of state and trajectory stabilization problems for nonlinear dynamical systems. Our approach is able to synthesize relatively simple polynomial control Lyapunov functions, and in that process replace the MPC using a guaranteed and computationally less expensive controller.

Path-Following through Control Funnel Functions

“…Synthesizing Funnels: We adapted a recently developed demonstrator-based learning framework to synthesize control funnel functions for given sets I, G, and S [7]. We note that it is possible to use SOS programming to design feedback and funnel function [5] to address the control design problem.…”

“…To combat this, Majumdar et al use LQR controllers and their associated Lyapunov functions for the linearization of the dynamics as good initial seed solutions [5]. In contrast, recent work by some of the authors remove the bilinearity by using a demonstrator in the form of a MPC controller [7]. Furthermore, this approach avoids local saddle points and has a fast convergence guarantee.…”

Data-Driven Computation of Robust Control Invariant Sets With Concurrent Model Selection

“…The latter article proposes an identification method that uses mixed-integer programming to identify a piecewise linear model with a bound on the disturbance for formal synthesis. The works in [31] and [32] discuss data-driven stability analysis centered around Lyapunov functions but without any consideration for control synthesis. What is clear, in all of the abovementioned articles, however, is that system identification and control synthesis are undertaken separately, and the identified model is not necessarily "optimal" for control synthesis.…”

Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems

“…We are particularly interested in learning-based approaches that guarantee Lyapunov stability [21]. From that perspective, the bulk of previous work has focused on using learning to construct a Lyapunov function [31], [12], [30], or to assess the region of attraction for a Lyapunov function [10], [7]. * One limitation of previous work is the learning is conducted over the full-dimensional state space, which can be data inefficient.…”