Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique

Neural-network-based adaptive optimal tracking control scheme for discrete-time nonlinear systems with approximation errors

“…To overcome the difficulty, many approximation methods are proposed to obtain optimal tracking control law [2][3][4][5]. Among these approximate approaches, adaptive dynamic programming (ADP) algorithm, proposed by Werbos [6,7], has played an important role in seeking approximate solutions of dynamic programming problems as a way to solve the computational issue forward-in-time [8][9][10][11][12][13][14]. There are several synonyms used for ADP including "adaptive critic designs" [15], "adaptive dynamic programming" [16,17], "approximate dynamic programming" [18,19], "neural dynamic programming" [20], "neuro-dynamic programming" [21], and "reinforcement learning" [22].…”

Online optimal control of unknown discrete-time nonlinear systems by using time-based adaptive dynamic programming

“…A class of RL-based adaptive optimal controllers, called approximate/adaptive dynamic programming (ADP), was first developed by Werbos [5,6]. Extensions of the RLbased controllers to DT systems have been considered by many researchers [7][8][9][10][11][12][13][14][15][16][17][18][19][20]. In [7], the authors attempted to solve the DT nonlinear optimal control problem offline using ADP approaches and neural networks by assuming that there are no NN reconstruction errors.…”

“…The work of [9] analyzed the convergence of unknown DT nonlinear systems using offlinetrained neural networks, but this method introduced the Lebesgue integral [7], which required data of a subset of the plant, in the tuning law and thus spent too much time on off-line training. In [20], the authors developed one way to control the unknown DT nonlinear systems using globalized dual heuristic programming, and others employed the single network dual heuristic dynamic programming (SN-DHP) technique in the ADP algorithm in [19]. Both of them introduced the gradient-based adaptation tuning law instead of the way in [9].…”

“…Both of them introduced the gradient-based adaptation tuning law instead of the way in [9]. However, without using recorded system data, iterations were needed in the tuning law [19,20] and the critic NN and actor NN could not be updated with respect to time at each sampling interval. Moreover, although [9,20,21] constructed a NN to identify the unknown system dynamics, they assumed that the NN identification error approached to zero, and thus the effects of the estimation error on the convergence of the actor-critic algorithms were not considered.…”

Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults

“…Moreover, for addressing the nonlinear optimal control problem [5][6][7][8][9][10][11][12][13][14], it can be converted to solve a Hamilton-Jacobi-Bellman (HJB) equation instead of the ARE. However, it is difficult or even impossible to solve the HJB equation.…”