Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis

Wei, Qinglai; Lewis, Frank L.; Liu, Derong; Song, Ruizhuo; Lin, Hanquan

doi:10.1109/tsmc.2016.2623766

Cited by 142 publications

(44 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, due to the fact that the nonlinear partial difference equations need to be solved especially for discrete‐time system, the nonlinear Hamilton‐Jacobi‐Bellman (HJB) equation of nonlinear system is more difficult to solve than Riccati equation. Fortunately, adaptive dynamic programming (ADP) as an effective approach to solve nonlinear optimal control problem has received much interest . Not only the stability of system is guaranteed but also the performance index is minimized by ADP.…”

Section: Introductionmentioning

confidence: 99%

“…Fortunately, adaptive dynamic programming (ADP) as an effective approach to solve nonlinear optimal control problem has received much interest. [12][13][14][15] Not only the stability of system is guaranteed but also the performance index is minimized by ADP. Normally, ADP approaches are classified into several schemes, such as heuristic dynamic Int J Robust Nonlinear Control.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Multigradient recursive reinforcement learning NN control for affine nonlinear systems with unmodeled dynamics

Bai

Zhang

Zhou

et al. 2019

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Summary In this paper, an adaptive reinforcement learning approach is developed for a class of discrete‐time affine nonlinear systems with unmodeled dynamics. The multigradient recursive (MGR) algorithm is employed to solve the local optimal problem, which is inherent in gradient descent method. The MGR radial basis function neural network approximates the utility functions and unmodeled dynamics, which has a faster rate of convergence than that of the gradient descent method. A novel strategic utility function and cost function are defined for the affine systems. Finally, it concludes that all the signals in the closed‐loop system are semiglobal uniformly ultimately bounded through differential Lyapunov function method, and two simulation examples are presented to demonstrate the effectiveness of the proposed scheme.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

Multigradient recursive reinforcement learning NN control for affine nonlinear systems with unmodeled dynamics

Bai

Zhang

Zhou

et al. 2019

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

show abstract

“…Then, several names emerged, eg, approximate dynamic programming and asymptotic dynamic programming. Iterative methods are widely used in ADP to obtain solutions of the Bellman equation indirectly . Adaptive dynamic programming can be divided into many categories, such as heuristic dynamic programming (HDP) dual heuristic dynamic programming (DHP), action dependent DHP (also called Q‐learning), globalized DHP, and so on.…”

Section: Introductionmentioning

confidence: 99%

“…Iterative methods are widely used in ADP to obtain solutions of the Bellman equation indirectly. [10][11][12] Adaptive dynamic programming can be divided into many categories, such as heuristic dynamic programming (HDP) [13][14][15] dual heuristic dynamic programming (DHP), 16 action dependent DHP (also called Q-learning 17 ), globalized DHP, 18 and so on. Adaptive dynamic programming has been widely used in real-world applications.…”

Section: Introductionmentioning

confidence: 99%

A novel triggering condition of event‐triggered control based on heuristic dynamic programming for discrete‐time systems

Wang

Wei

Liu

2018

Optim Control Appl Methods

Self Cite

View full text Add to dashboard Cite

Summary In this paper, an event‐triggered heuristic dynamic programming algorithm for discrete‐time nonlinear systems with a novel triggering condition is studied. Different from traditional heuristic dynamic programming algorithms, the control law in this algorithm will only be updated when the triggering condition is satisfied to reduce the computational burden. Three neural networks are employed, which are model network, action network, and critic network. Model functions, control laws, and value functions are estimated using neural networks, respectively. The main contribution of this algorithm is the novel triggering condition with simpler form and fewer assumptions. Additionally, a proof of the stability for discrete‐time systems using Lyapunov technique is given. Finally, two simulations are shown to verify the effectiveness of the developed algorithm.

show abstract

“…The proposed control scheme combined the backstepping technique, DSC approach, and optimal control theory; thus, computational complexity can be greatly reduced without losing optimal control performance, and the appearance of unknown information will be avoided in the recursive design. Although other authors(–) also proposed adaptive optimal control methods based on ADP, the aforementioned results are applied only to discrete‐time systems and the optimal control methods(–) cannot solve the optimal control problem for continuous‐time systems; this paper proposed an adaptive‐dynamic‐programming‐based fuzzy controller that not only can have robustness to unknown time delay for nonlinear continuous‐time systems but also can guarantee the tracking error to an arbitrarily small residual set and achieve optimal control objective.…”

Section: Introductionmentioning

confidence: 99%

Adaptive‐dynamic‐programming‐based fuzzy control for triangular structure nonlinear uncertain systems with unknown time delay

Sui

Tong

Sun

2017

Optim Control Appl Methods

View full text Add to dashboard Cite

Summary This paper develops a novel fuzzy adaptive backstepping dynamic surface control scheme for a class of strict‐feedback nonlinear systems with unknown time delay based on adaptive dynamic programming. The whole control scheme is composed of 2 parts, ie, feedforward control and feedback optimal control. Fuzzy logic systems are used to identify nonlinear systems. The control design method, which combined the backstepping design technique and the dynamic surface control approach, is applied to convert triangular structure systems to affine nonlinear systems; it not only can solve the problem of “explosion of complexity” but also can avoid the appearance of unknown terms. The closed‐loop system is guaranteed to be uniformly ultimately bounded based on the Lyapunov and optimal control theory, and the system output can track the desired reference signal in an optimal manner. The simulation results of numerical examples are presented to show the effectiveness of the proposed approach.

show abstract

Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis

Cited by 142 publications

References 51 publications

Multigradient recursive reinforcement learning NN control for affine nonlinear systems with unmodeled dynamics

Multigradient recursive reinforcement learning NN control for affine nonlinear systems with unmodeled dynamics

A novel triggering condition of event‐triggered control based on heuristic dynamic programming for discrete‐time systems

Adaptive‐dynamic‐programming‐based fuzzy control for triangular structure nonlinear uncertain systems with unknown time delay

Contact Info

Product

Resources

About