Adaptive optimal tracking controls of unknown multi-input systems based on nonzero-sum game theory

Lv, Yongfeng; Ren, Xuemei; Na, Jing

doi:10.1016/j.jfranklin.2019.07.022

Cited by 28 publications

(15 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To realize the optimal tracking control design, we decompose the control input u into two parts [18,21] as…”

Section: Online Dynamic Tracking Controlmentioning

confidence: 99%

“…en, the approximated Hamiltonian function can be derived as 0 ≜ H(x, u, W) � e T Qe + u T e Ru e + W T ∇σ(e) _ e � e c . (21) For training the critic NN to obtain the control action, it is expected to estimate W to minimize the objective function E � (1/2)e T c e c . Hence, the gradient descent algorithm can be used to update the critic NN weights W by…”

Section: Neural Network Approximationmentioning

confidence: 99%

“…However, the OTCP is more challenging to address than the regulation problem, since its solution is usually composed of a feedforward action to guarantee the perfect tracking and a feedback action to stabilize the closed-loop system dynamics [19]. For linear systems, the solution of OTCP is calculated by addressing the Riccati equations [20], while for nonlinear systems, the existing solution for the OTCP can be derived with a feedforward term by employing the dynamics inversion and a feedback term by calculating a complex HJB equation [16,21]. However, it is well known that deriving the solution of OTCP is typically intractable, especially for the online tracking control.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Online Optimal Control of Robotic Systems with Single Critic NN‐Based Reinforcement Learning

Long

Wang

2021

Complexity

View full text Add to dashboard Cite

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method.

show abstract

“…To realize the optimal tracking control design, we decompose the control input u into two parts [18,21] as…”

Section: Online Dynamic Tracking Controlmentioning

confidence: 99%

Section: Neural Network Approximationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online Optimal Control of Robotic Systems with Single Critic NN‐Based Reinforcement Learning

Long

Wang

2021

Complexity

View full text Add to dashboard Cite

show abstract

“…In [24], fuzzy logic models, which are similar to T−S fuzzy models, were combined with ADP to solve the NZS games issues. For the unknown multi−input system, a three−layer NN identifier, reinforcement learning scheme and NZS game theory were utilized together to solve the optimal tracking control issue [25]. In [26], a model network was designed to identify the system whose dynamic is not known.…”

Section: Introductionmentioning

confidence: 99%

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

et al. 2020

View full text Add to dashboard Cite

In this paper, a novel online method for multi-player non-zero-sum (NZS) differential games of nonlinear partially unknown continuous time (CT) systems with control constraints is developed based on neural networks (NN). The issue of multi-player NZS games with saturated actuator is elaborately analyzed and the unknown dynamics model is learned by applying identifier NN. Different from using the standard identifier-actor-critic framework of adaptive dynamic programming (ADP), the proposed method uses only identifier networks and critic networks for all the players to solve the coupled Hamilton-Jacobi (HJ) equations for multi-player NZS games, which could effectively simplify the algorithm and save computing resources. Moreover, a tuning law which utilizes the gradient descent method is designed for each critic network. Meanwhile, to remove the requirement for the initial stabilizing control, a novel stability term is designed to ensure the system stability during the training phase of the critic NN. By the means of Lyapunov approach, it is proven that the system states, the critic network weight estimation errors and the obtained control are all uniformly ultimately bounded (UUB). Finally, two numerical examples are simulated to illustrate the validity of the developed method for multi-player NZS games with control constraints. INDEX TERMS Adaptive critic designs, adaptive dynamic programming, control constraints, multi-player, non-zero-sum games.

show abstract

“…In the work of Vamvoudakis et al, IRL and actor/critic framework were combined, skipping the step of solving HJB and Bellman equations related to partially unknown parameters. An ADP algorithm was proposed based on identifier‐critic RL and nonzero‐sum game theory in the work of Lv et al Zargarzadeh et al took optimality and stability into account and proposed NN ‐based learning controller for nonlinear systems in strict‐feedback form. Based on ADP idea, output feedback fuzzy optimal scheme was designed in the work of Sun et al, where fuzzy universal approximator was introduced to replace NNs.…”

Section: Introductionmentioning

confidence: 99%

Adaptive optimal dynamic surface control of strict‐feedback nonlinear systems with output constraints

Zhang

2019

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Summary In this paper, an adaptive optimal control strategy is proposed for a class of strict‐feedback nonlinear systems with output constraints by using dynamic surface control. The controller design procedure is divided into two parts. One is the design of feedforward controller and the other is the design of optimal controller. To guarantee the satisfaction of output constraints in feedforward controller, nonlinear mapping is utilized to transform the constrained system into an unconstrained system. Neural‐network based adaptive dynamic programming algorithm is employed to approximate the optimal cost function and the optimal control law. By theoretical analysis, all the signals in the closed‐loop system are proved to be semi‐globally uniformly ultimately bounded and the output constraints are not violated. A numerical example illustrates the effectiveness of the proposed scheme.

show abstract

Adaptive optimal tracking controls of unknown multi-input systems based on nonzero-sum game theory

Cited by 28 publications

References 42 publications

Online Optimal Control of Robotic Systems with Single Critic NN‐Based Reinforcement Learning

Online Optimal Control of Robotic Systems with Single Critic NN‐Based Reinforcement Learning

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

Adaptive optimal dynamic surface control of strict‐feedback nonlinear systems with output constraints

Contact Info

Product

Resources

About