Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

Abstract

In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

“…The result would be compared to the result which is obtained by using the Gauss-Newton method [18] [19]. Hence, the calculation procedure in the IOCPE could be simplified.…”

“…In addition, the optimal output solution obtained from the IOCPE algorithm has been improved by using the weighted output residual [16], which is introduced into the model cost function, and the output matching scheme [17], where the adjusted parameter is introduced into the model output. Moreover, the application of the approaches on the least-square and the Gauss-Newton with the principle of model-reality differences, which omits from using the adjusted parameters, enhance the practical usage of the IOCPE algorithm for delivering the optimal solution of the original optimal control problem [18] [19].…”

A Gauss-Newton Approach for Nonlinear Optimal Control Problem with Model-Reality Differences

“…On the basis of this, it is highlighted that applying the least-square updating scheme for solving discrete-time nonlinear optimal control problems, both for deterministic and stochastic cases, are well-presented. See [24] for more details on stochastic case.…”