No abstract

No abstract

The extended infinite horizon optimal control problem of continuous time linear systems with unknown dynamics is investigated in this paper. This optimal control problem can be solved using the corresponding extended algebraic Riccati equation. A new policy iteration algorithm is proposed to approximate the solution of the extended algebraic Riccati equation when the system dynamics are known. The convergence of the proposed algorithm is proved. Based on the proposed policy iteration algorithm, an online adaptive dynamic programming (ADP) algorithm is developed to find the solution to the extended infinite horizon optimal control problem of unknown continuous time linear systems. The convergence of the online ADP algorithm is analyzed. Finally, two simulation examples are given to demonstrate the effectiveness of the developed online ADP algorithm.

This paper investigates finite-horizon optimal control problem of completely unknown discrete-time linear systems. The completely unknown here refers to that the system dynamics are unknown. Compared with infinitehorizon optimal control, the Riccati equation (RE) of finite-horizon optimal control is time-dependent and must meet certain terminal boundary constraints, which brings the greater challenges. Meanwhile, the completely unknown system dynamics have also caused additional challenges. The main innovation of this paper is the developed cyclic fixed-finite-horizon-based Q-learning algorithm to approximate the optimal control input without requiring the system dynamics. The developed algorithm main consists of two phases: the data collection phase over a fixed-finite-horizon and the parameters update phase. A least-squares method is used to correlate the two phases to obtain the optimal parameters by cyclic. Finally, simulation results are given to verify the effectiveness of the proposed cyclic fixed-finite-horizon-based Q-learning algorithm.2010 Mathematics Subject Classification. Primary: 58F15, 58F17; Secondary: 53C35.