A Bayesian Perspective on Stochastic Neurocontrol

Abstract

Control design for stochastic uncertain nonlinear systems is traditionally based on minimizing the expected value of a suitably chosen loss function. Moreover, most control methods usually assume the certainty equivalence principle to simplify the problem and make it computationally tractable. We offer an improved probabilistic framework which is not constrained by these previous assumptions, and provides a more natural framework for incorporating and dealing with uncertainty. The focus of this paper is on developing this framework to obtain an optimal control law strategy using a fully probabilistic approach for information extraction from process data which does not require detailed knowledge of system dynamics. Moreover the proposed control method framework allows handling the problem of input-dependent noise. A basic paradigm is proposed and the resulting algorithm is discussed. The proposed probabilistic control method is demonstrated theoretically on two classes of nonlinear discrete time systems: the affine and the general class. A nonlinear simulation example is also provided to validate theoretical development.

Generalised Probabilistic Control Design for Uncertain Stochastic Control Systems

“…The main advantage of the FPD is that it provides an explicit form of the randomised optimal controller. However, since the evaluation of the randomised optimal controller involves multivariate integration steps, as can be seen from (1), which needs to be computed by backward recursion the problem renders to be nontrivial and computationally very intensive. To overcome the difficulties arising in the FPD, a probabilistic Dual Heuristic Programming (DHP) adaptive critic method was proposed in [8,9].…”

Non standard Ricatti solution and linear quadratic pinning control

“…However, in most control applications it is practically impossible to obtain an accurate mathematical model of the dynamics. Therefore, researchers have recently considered modeling the conditional distributions of the stochastic systems rather than relying on the single estimate of their parameters [8], [14].…”

“…However deriving the control law based on heuristic certainty equivalence (HCE) control principle has been shown to lead to an inadequate transient response because of the initial uncertainty involved in the unknown parameters [8], [14]. The optimal control of linear systems with uncertain parameters and quadratic cost functions has then been formulated in the context of guaranteed cost control [4], [6].…”

Fully probabilistic control for stochastic nonlinear control systems with input dependent noise

“…As a result, several groups of control methods have been developed. Examples of the so far developed groups of control methods are: 1) stochastic distribution control, which is concerned with the problem of designing a controller so that the pdf of the system output follows a pre-specified ideal distribution (Wang and Afshar, 2009); 2) closed loop pdf control (Kárný, 1996;Kárný and Guy, 2006), where the selected control influences the closed loop description of the system under consideration; and 3) the control of the tracking error pdf (Herzallah and Lowe, 2008). This paper is concerned with extending the second group of control methods, known as fully probabilistic design (FPD).…”