Disturbance Decoupling for Gradient-Based Multi-Agent Learning With Quadratic Costs

Abstract

Motivated by applications of multi-agent learning in noisy environments, this paper studies the robustness of gradientbased learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player's actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled-i.e., such players' actions are completely unaffected by the injected disturbance. We provide necessary and sufficient conditions to guarantee this property for games with quadratic cost functions, which encompass quadratic one-shot continuous games, finitehorizon linear quadratic (LQ) dynamic games, and bilinear games. Specifically, disturbance decoupling is characterized by both algebraic and graph-theoretic conditions on the learning dynamics, the latter is obtained by constructing a game graph based on gradients of players' costs. For LQ games, we show that disturbance decoupling imposes constraints on the controllable and unobservable subspaces of players. For two player bilinear games, we show that disturbance decoupling within a player's action coordinates imposes constraints on the payoff matrices. Illustrative numerical examples are provided.

Robust Networked Multiagent Optimization: Designing Agents to Repair Their Own Utility Functions

“…It is often the case that the system designer can focus exclusively on the design of utility functions without explicitly considering the dynamics that will ultimately drive the collective behavior to such equilibria (de Jong and Uetz, 2020; Li and Marden, 2013). The reason for this decomposition stems from the fact that if the designed agent utility functions posses a desirable structure, e.g., form a weakly acyclic or potential game, then a system operator can appeal to off-the-shelf distributed learning algorithms that can be employed to drive the collective behavior to an equilibrium of interest, e.g., fictitious play or log-linear learning (Alós-Ferrer and Netzer, 2010;Candogan et al, 2013;Fudenberg and Levine, 1998;Li et al, 2021). Accordingly, the design of distributed control algorithms for engineered multi-agent systems can be recast as the problem of designing utility functions for the agents such that the induced games have desirable equilibrium properties.…”