Near-Optimal Control Strategy in Leader-Follower Networks: A Case Study for Linear Quadratic Mean-Field Teams

Abstract

In this paper, a decentralized stochastic control system consisting of one leader and many homogeneous followers is studied. The leader and followers are coupled in both dynamics and cost, where the dynamics are linear and the cost function is quadratic in the states and actions of the leader and followers. The objective of the leader and followers is to reach consensus while minimizing their communication and energy costs. The leader knows its local state and each follower knows its local state and the state of the leader. The number of required links to implement this decentralized information structure is equal to the number of followers, which is the minimum number of links for a communication graph to be connected. In the special case of leaderless, no link is required among followers, i.e., the communication graph is not even connected. We propose a near-optimal control strategy that converges to the optimal solution as the number of followers increases. One of the salient features of the proposed solution is that it provides a design scheme, where the convergence rate as well as the collective behavior of the followers can be designed by choosing appropriate cost functions. In addition, the computational complexity of the proposed solution does not depend on the number of followers. Furthermore, the proposed strategy can be computed in a distributed manner, where the leader solves one Riccati equation and each follower solves two Riccati equations to calculate their strategies. Two numerical examples are provided to demonstrate the effectiveness of the results in the control of multi-agent systems.

MinMax Mean-Field Team Approach for a Leader–Follower Network: A Saddle-Point Strategy

“…However, these differences do not add much complexity to the convergence proof because the Riccati equations (5) do not depend on the number of followers according to Assumption 3. Hence, the rate of convergence with respect to the number of followers is 1/n, similar to [10,Theorem 2]. This leads to the following theorem.…”

“…Since the dynamic equations ( 1) and ( 2), cost function ( 3) and the saddle-point strategies ( 6), ( 7) and ( 8) are bounded and continuous in xt , it results that ( 8) is an approximate saddle-point strategy. The reader is referred to [10] for a detailed proof in the context of optimal control, which is similar, to a great extent, to the convergence proof of MinMax control problem considered in this subsection, but note that the Riccati equations here are different and the relative errors defined in [10] are of intermittent nature. However, these differences do not add much complexity to the convergence proof because the Riccati equations (5) do not depend on the number of followers according to Assumption 3.…”

“…In contrast, mean-field team [7] focuses on the finite-population model, where the effect of each individual player is not necessarily negligible. For linear quadratic mean-field teams, a transformation-based technique was first proposed in [8] and further extended in [7,9,10], that is similar in spirit to the completion-of-square approach in mean-field-type game [11,12]. In general, mean-field-type game may be viewed as a special case of mean-field team, where the information structure is mean-field sharing and the number of players is infinite.…”

Deep Structured Teams in Arbitrary-Size Linear Networks: Decentralized Estimation, Optimal Control and Separation Principle

“…Furthermore, we demonstrate in [9] that today's most common feed-forward deep neural networks (i.e., those with rectified linear unit activation function) may be viewed as a special case of deep structured teams, where layers are time steps and neurons are simple integrator agents whose goal is to collaborate in order to minimize a common loss (cost) function. For more applications of deep structured models, the reader is referred to reinforcement learning [8], [13], [14], nonzerosum game [12], [15], minmax optimization [17], leaderfollowers [18], [19], epidemics [20], smart grids [21], meanfield teams [22]- [25], and networked estimation [26], [27].…”

A Mean-Field Team Approach to Minimize the Spread of Infection in a Network

“…In addition, we raise a practical question that when the solution of an infinite-population network constructs a meaningful approximation for the finitepopulation one. Inspired by existing techniques for meanfield teams [19], [20], [21], [22], [23], [24], we first compute the optimal solution of a finite-population network for the case where the empirical distribution of infected nodes is observable. Next, we derive an infinite-population Bellman equation that requires no observation of infected nodes, and identify a stability condition under which the solution of the infinite-population network constitutes a near-optimal solution for the finite-population one.…”