Fully probabilistic control for stochastic nonlinear control systems with input dependent noise

Abstract

Robust controllers for nonlinear stochastic systems with functional uncertainties can be consistently designed using probabilistic control methods. In this paper a generalised probabilistic controller design for the minimisation of the Kullback-Leibler divergence between the actual joint probability density function (pdf) of the closed loop control system, and an ideal joint pdf is presented emphasising how the uncertainty can be systematically incorporated in the absence of reliable systems models. To achieve this objective all probabilistic models of the system are estimated from process data using mixture density networks (MDNs) where all the parameters of the estimated pdfs are taken to be state and control input dependent. Based on this dependency of the density parameters on the input values, explicit formulations to the construction of optimal generalised probabilistic controllers are obtained through the techniques of dynamic programming and adaptive critic methods. Using the proposed generalised probabilistic controller, the conditional joint pdfs can be made to follow the ideal ones. A simulation example is used to demonstrate the implementation of the algorithm and encouraging results are obtained.

Generalised Probabilistic Control Design for Uncertain Stochastic Control Systems

“…The approach followed in this paper is pragmatic and fully probabilistic. It is radically different to the state of the art control design methods [11][12][13][14][15][16][17] which are concerned with the minimisation of objective functions that are confined to be either the mean value or variance of the stochastic output.…”

“…Theorem 1. Using the elements defined in Equations (11), (12), (13), (14), and (15) in Equation (6) yields the following optimal performance index:…”

“…For the system defined in Equation (11), and the elements in Equations (12), (13), (14), and (15), the numerator of Eq. (27) evaluates to,…”

Discrete fully probabilistic design: towards a control pipeline for the synthesis of policies from examples

“…In the context of control, the approach has been leveraged in e.g. Kárný (1996); ; Kárný and Kroupa (2012); Herzallah (2015); Guy et al (2018); Pegueroles and Russo (2019) for the design of randomized control policies that enable tracking of a given target behavior. In these papers, the tracking problem is tackled by setting-up an unconstrained optimization problem.…”

On robust stability of fully probabilistic control with respect to data-driven model uncertainties

“…The fully probabilistic control algorithm, which was originally introduced in the seminal work [8], belongs to the family of stochastic control algorithms, with the main difference that it selects randomized control laws that make the entire joint distribution of closed-loop variables as close as possible (in the sense of the Kulback-Leibler divergence) to their desired distribution. See also [9], [10], [11] for recent developments on this topic and [12].…”