H∞ tracking control via variable gain gradient descent‐based integral reinforcement learning for unknown continuous time non‐linear system

Abstract

Optimal tracking of continuous‐time non‐linear systems has been extensively studied in literature. However, in several applications, absence of knowledge about system dynamics poses a severe challenge in solving the optimal tracking problem. This has found growing attention among researchers recently, and integral reinforcement learning based method augmented with actor neural network (NN) have been deployed to this end. However, very few studies have been directed to model‐free H∞ optimal tracking control that helps in attenuating the effect of disturbances on the system performance without any prior knowledge about system dynamics. To this end, a recursive least square‐based parameter update was recently proposed. However, gradient descent‐based parameter update scheme is more sensitive to real‐time variation in plant dynamics. Experience replay (ER) technique has been shown to improve the convergence of NN weights by utilising past observations iteratively. Motivated by these, this study presents a novel parameter update law based on variable gain gradient descent and ER technique for tuning the weights of critic, actor and disturbance NNs. The presented update law leads to improved model‐free tracking performance under L2‐bounded disturbance. Simulation results are presented to validate the presented update law.

Regulation With Guaranteed Convergence Rate for Continuous-Time Systems With Completely Unknown Dynamics in the Presence of Disturbance

“…Moreover, IRLbased event-triggered ADP is introduced in [42] to cut the need for drift dynamics and to control a CT nonlinear system with saturated input. In [43], the authors considered a CT neural network parameter update law based on variable gain gradient descent augmented with robust terms for model-free IRL-based H ∞ optimal tracking control problem of a CT nonlinear system with unknown dynamics for disturbance rejection.…”

Finite‐horizon H∞ tracking control for discrete‐time linear systems

“…Off-policy RL algorithms have been developed to achieve the H ∞ tracking control of continuous-time (CT) nonlinear systems with unknown system dynamics. 20,21 Q-learning 22 is one of the RL algorithms that learns the optimal Q-function. The Q-function explicitly contains the control actions and evaluates all possible actions for each state.…”

“…RL‐based techniques have also been applied to provide the model‐free solution to the $$$ {H}_{\infty } $$$ tracking control problem. Off‐policy RL algorithms have been developed to achieve the $$$ {H}_{\infty } $$$ tracking control of continuous‐time (CT) nonlinear systems with unknown system dynamics 20,21 . Q‐learning 22 is one of the RL algorithms that learns the optimal Q‐function.…”

“…However, the above‐mentioned RL‐based $$$ {H}_{\infty } $$$ tracking control methods 15‐21,27‐31 concern the infinite‐horizon cases. These methods can only achieve asymptotic $$$ {H}_{\infty } $$$ tracking control, which means that the theoretical regulation time tends to be infinite.…”

“…The innovation of this paper can be given by comparing it with the existing works. Firstly, the discount factor which plays a key role in infinite‐horizon $$$ {H}_{\infty } $$$ tracking controller design 15‐21,27‐31 is not required in the developed finite‐horizon $$$ {H}_{\infty } $$$ tracking control method. The discount factor will affect the tracking error, whereas the developed method in this paper will not encounter this problem.…”