Smoothing Solution for Discrete-Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

Abstract

In this chapter, the performance of the integrated optimal control and parameter estimation (IOCPE) algorithm is improved using a modified fixed-interval smoothing scheme in order to solve the discrete-time nonlinear stochastic optimal control problem. In our approach, a linear model-based optimal control problem with adding the adjustable parameters into the model used is solved iteratively. The aim is to obtain the optimal solution of the original optimal control problem. In the presence of the random noise sequences in process plant and measurement channel, the state dynamics, which is estimated using Kalman filtering theory, is smoothed in a fixed interval. With such smoothed state estimate sequence that reduces the output residual, the feedback optimal control law is then designed. During the computation procedure, the optimal solution of the modified model-based optimal control problem can be updated at each iteration step. When convergence is achieved, the iterative solution approaches to the correct optimal solution of the original optimal control problem, in spite of model-reality differences. Moreover, the convergence of the resulting algorithm is also given. For illustration, optimal control of a continuous stirred-tank reactor problem is studied and the result obtained shows the efficiency of the approach proposed.

Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

“…The linear model-based optimal control problem, which is simplified from Problem (P) and is referred to as Problem (M), is defined by By running the approach proposed, the simulation result is shown in Table 1, where it is compared to the result of the filtering solution [15]. It can be seen that the iteration number of the approach proposed is more than the iteration number of filtering model, and the final cost of the approach proposed is greater than the final cost of filtering model.…”

“…In recent years, using the linear optimal control model with model-reality differences in solving the nonlinear optimal control problem, especially for discrete-time nonlinear stochastic optimal control problem, is proposed [14] [15] [16] [17]. Such method is known as the integrated optimal control and parameter estimation (IOCPE) algorithm.…”

“…Besides, the applications of the IOCPE algorithm in providing the expectation solution as well as the filtering solution of the discrete-time nonlinear stochastic optimal control problem have been well-demonstrated [14] [15]. In addition, the optimal output solution obtained from the IOCPE algorithm has been improved by using the weighted output residual [16], which is introduced into the model cost function, and the output matching scheme [17], where the adjusted parameter is introduced into the model output.…”

Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

“…Recently, the integrated optimal control and parameter estimation (ICOPE) algorithm, which solves the linear model-based optimal control problem iteratively, is proposed in the literature [18] [19]. The purpose of this algorithm is to provide the optimal solution of the discrete time nonlinear stochastic optimal control problem with the different structure and parameters.…”