Neighbor Q‐learning based consensus control for discrete‐time multi‐agent systems

Abstract

The neighbor Q-learning based consensus control algorithm is developed for discrete-time multi-agent systems in this article. To realize the proposed algorithm, a new actor-critic architecture is employed for each agent. The critic network of each agent approximates its Q-function while the actor network produces control signal by minimizing the Q-function. Considering the distribution metrics of the systems, the neighbors' Q-functions of each agent are applied to the update procedure of the critic network to stabilize the learning process and avoid the overestimation problem. The convergence properties and stability analysis for the proposed algorithm are provided. Different discount factors corresponding to various topology structures are discussed in the convergence analysis section. The accurate system model is nonessential for the algorithm which is too intricate to build up for practical systems. Finally, three simulation examples including different discount factors are conducted to demonstrate the effectiveness of the consensus control algorithm.

Data‐based learning control for optimization of nonlinear systems

“…The fifth group of papers 24‐27 discusses data‐based control for distributed control systems. A mission‐driven control scheme, including a consensus‐based near‐optimal formation controller and a finite‐time precise formation controller, is proposed aiming at different requirements of unmanned aerial vehicle swarm 24 .…”

“…The neural network adaptive formation control of a class of second‐order nonlinear systems with unmodeled dynamics is investigated, where the control law merely depends on the relative bearings between neighboring agents 25 . The neighbor Q‐learning based consensus control algorithm is developed for discrete‐time multiagent systems 26 . The fault‐tolerate containment control problem is considered for stochastic nonlinear multiagent systems in the presence of input saturation and sensor faults 27 …”

“…22 A singularity-free online neural network-based sliding mode control method is proposed to realize the fixed-wing perch maneuver. 23 The fifth group of papers [24][25][26][27] discusses data-based control for distributed control systems. A mission-driven control scheme, including a consensus-based near-optimal formation controller and a finite-time precise formation controller, is proposed aiming at different requirements of unmanned aerial vehicle swarm.…”

“…25 The neighbor Q-learning based consensus control algorithm is developed for discrete-time multiagent systems. 26 The fault-tolerate containment control problem is considered for stochastic nonlinear multiagent systems in the presence of input saturation and sensor faults. 27 The sixth group of papers [28][29][30] considers applications of data-based learning methods to industrial processes.…”