Suboptimal stabilizing controllers for linearly solvable system

Abstract

Abstract-This paper presents a novel method to synthesize stochastic control Lyapunov functions for a class of nonlinear, stochastic control systems. In this work, the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation is transformed into a linear partial differential equation for a class of systems with a particular constraint on the stochastic disturbance. It is shown that this linear partial differential equation can be relaxed to a linear differential inclusion, allowing for approximating polynomial solutions to be generated using sum of squares programming. It is shown that the resulting solutions are stochastic control Lyapunov functions with a number of compelling properties. In particular, a-priori bounds on trajectory suboptimality are shown for these approximate value functions. The result is a technique whereby approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems.

Controller synthesis for stochastic systems with persistent noise via semi-definite programming

“…Applying Schur's complement to (20), we see that inequality (13) is satisfied for the closed loop system. Moreover, inequality (19) assures that (12) is also satisfied.…”

“…Controller synthesis with performance bounds was also considered in [12] and [17] and controller synthesis for discontinuous dynamical systems was studied in [18]. In the context of stochastic systems, [19] and [20] are the only attempts using SOS programming to synthesize controllers, where the authors use a logarithmic transformation to obtain a linear version of the Hamilton-Jacobi-Bellman (HJB) equation.…”

Linearly Solvable Stochastic Control Lyapunov Functions

“…Proof. Recall that from the proof of Theorem 23, all conditions in Definition 29 are satisfied by V u except (24). To show that V u satisfies (24), rearrange (4) to yield the following…”

“…These previous works focused on path planning, rather than stabilization, and did not include the stability analysis or suboptimality guarantees presented in this paper. A short version of this work appeared in [24] which included less details and did not include the extension in section 5.…”