Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

Abstract

In this paper, a computational approach is proposed for solving the discrete-time nonlinear optimal control problem, which is disturbed by a sequence of random noises. Because of the exact solution of such optimal control problem is impossible to be obtained, estimating the state dynamics is currently required. Here, it is assumed that the output can be measured from the real plant process. In our approach, the state mean propagation is applied in order to construct a linear model-based optimal control problem, where the model output is measureable. On this basis, an output error, which takes into account the differences between the real output and the model output, is defined. Then, this output error is minimized by applying the stochastic approximation approach. During the computation procedure, the stochastic gradient is established, so as the optimal solution of the model used can be updated iteratively. Once the convergence is achieved, the iterative solution approximates to the true optimal solution of the original optimal control problem, in spite of model-reality differences. For illustration, an example on a continuous stirred-tank reactor problem is studied, and the result obtained shows the applicability of the approach proposed. Hence, the efficiency of the approach proposed is highly recommended.

Construction and Update of an Online Ensemble Score Involving Linear Discriminant Analysis and Logistic Regression

“…a flow of data arriving continuously, one wishes to be able to update such an ensemble score when new data becomes available, without having to store all of the previously obtained data and without performing the entire analysis. To achieve this goal, stochastic approximation processes [7] [8] [9] can be used. In particular, processes that we have previously studied theoretically [10] [11] will be detailed in Section 2.…”

Application of Conjugate Gradient Approach for Nonlinear Optimal Control Problem with Model-Reality Differences

“…Recently, the integrated optimal control and parameter estimation (IOCPE) algorithm has been proposed [1] in solving the nonlinear optimal control problem, both for discrete time deterministic and stochastic cases (see for more detail in [2]- [9]). In essence, the concept of the IOCPE algorithm is come from the dynamic integrated system optimization and parameter estimation (DISOPE) algorithm, which was developed by [10].…”