Learning Multi-Agent Action Coordination via Electing First-Move Agent

Abstract

Learning to coordinate actions among agents is essential in complicated multi-agent systems. Prior works are constrained mainly by the assumption that all agents act simultaneously, and asynchronous action coordination between agents is rarely considered. This paper introduces a bi-level multi-agent decision hierarchy for coordinated behavior planning. We propose a novel election mechanism in which we adopt a graph convolutional network to model the interaction among agents and elect a first-move agent for asynchronous guidance. We also propose a dynamically weighted mixing network to effectively reduce the misestimation of the value function during training. This work is the first to explicitly model the asynchronous multi-agent action coordination, and this explicitness enables to choose the optimal first-move agent. The results on Cooperative Navigation and Google Football demonstrate that the proposed algorithm can achieve superior performance in cooperative environments. Our code is available at https://github.com/Amanda-1997/EFA-DWM.

Learning to Collaborate by Grouping: A Consensus-Oriented Strategy for Multi-Agent Reinforcement Learning

“…Some works [3,8] use bisimulation metrics to extract the latent embeddings from observations. [19,1,31,30] attempt to learn action representations to assist multi-agent policy learning. [37,16] propose represent underlying messages to conduct effective communication in MAS.…”

Cooperative Multi-Agent Learning in a Complex World: Challenges and Solutions

“…• Various applications of cooperative multi-agent learning, including (Ruan et al 2022;Xu et al 2020;Wang et al 2021;Chen et al 2021). We will also discuss future directions for research in this field.…”