Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

Singh, Amanpreet; Jain, Tushar; Sukhbaatar, Sainbayar

doi:10.48550/arxiv.1812.09755

Cited by 21 publications

(39 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We benchmark our approach against two state-of-the-art, end-to-end communication learning baselines, namely the CommNet [26] and the IC3Net [25]. The results are presented in Figure 4.…”

Section: Baseline Comparison Resultsmentioning

confidence: 99%

“…MARL with Communication -Communication has been shown to further enhance the collective intelligence of learning agents in cooperative MARL problems [25]. In recent years, several studies have been concerned with the problem of learning communication protocols and languages to use among agents.…”

Section: Related Workmentioning

confidence: 99%

“…Predator-Prey (PP) [25] -The goal in this homogeneous environment is for N predator agents with limited vision to find a stationary prey and move to its location. The agents in this domain are homogeneous in their state, observation and action spaces and thus, all agents are of the same class.…”

Section: Evaluation Environmentsmentioning

confidence: 99%

See 2 more Smart Citations

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Seraj¹,

Wang²,

Paleja³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-agent teaming achieves better performance when there is communication among participating agents allowing them to coordinate their actions for maximizing shared utility. However, when collaborating a team of agents with different action and observation spaces, information sharing is not straightforward and requires customized communication protocols, depending on sender and receiver types. Without properly modeling such heterogeneity in agents, communication becomes less helpful and could even deteriorate the multi-agent cooperation performance. We propose heterogeneous graph attention networks, called HetNet, to learn efficient and diverse communication models for coordinating heterogeneous agents towards accomplishing tasks that are of collaborative nature. We propose a Multi-Agent Heterogeneous Actor-Critic (MAHAC) learning paradigm to obtain collaborative per-class policies and effective communication protocols for composite robot teams. Our proposed framework is evaluated against multiple baselines in a complex environment in which agents of different types must communicate and cooperate to satisfy the objectives. Experimental results show that HetNet outperforms the baselines in learning sophisticated multi-agent communication protocols by achieving ∼10% improvements in performance metrics.

show abstract

“…We benchmark our approach against two state-of-the-art, end-to-end communication learning baselines, namely the CommNet [26] and the IC3Net [25]. The results are presented in Figure 4.…”

Section: Baseline Comparison Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Seraj¹,

Wang²,

Paleja³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Value-based methods usually decompose the joint value function into individual value functions under the IGM (individual-global-max) principle, which guarantees the consistency between local action selection and joint action optimization (Sunehag et al, 2018;Rashid et al, 2020b;Son et al, 2019;Wang et al, 2021a;Rashid et al, 2020a). Other work also studies this problem from the perspective of agent roles and individuality (Wang et al, 2020a;b;Jiang & Lu, 2021) or communication learning (Singh et al, 2018;Das et al, 2019;Wang et al, 2020c). Compared to these methods, our work is built upon graph-based value decomposition, which explicitly models the interaction among agents.…”

Section: Related Workmentioning

confidence: 99%

Self-Organized Polynomial-Time Coordination Graphs

Yang¹,

Dong²,

Ren³

et al. 2021

Preprint

View full text Add to dashboard Cite

Coordination graph is a promising approach to model agent collaboration in multiagent reinforcement learning. It factorizes a large multi-agent system into a suite of overlapping groups that represent the underlying coordination dependencies. One critical challenge in this paradigm is the complexity of computing maximumvalue actions for a graph-based value factorization. It refers to the decentralized constraint optimization problem (DCOP), which and whose constant-ratio approximation are NP-hard problems. To bypass this fundamental hardness, this paper proposes a novel method, named Self-Organized Polynomial-time Coordination Graphs (SOP-CG), which uses structured graph classes to guarantee the optimality of the induced DCOPs with sufficient function expressiveness. We extend the graph topology to be state-dependent, formulate the graph selection as an imaginary agent, and finally derive an end-to-end learning paradigm from the unified Bellman optimality equation. In experiments, we show that our approach learns interpretable graph topologies, induces effective coordination, and improves performance across a variety of cooperative multi-agent tasks.

show abstract

“…CommNet can fuse and transfer information, which indirectly considers the global state output strategy [16]. Singh et al designed the IC3Net for the task of competitive mode, which controlled a communication-related binary gating function through the LSTM network gating mechanism and prevented the communication of multiple agents in a competitive relationship [17]. To better deal with the fully cooperative, partially observable multi-agent sequential decision problem, Foerster et al developed the DIAL, in which, communication information is passed through a discrete regularization unit (DRU) between the output of one network of agents and the input of another network of agents.…”

Section: B Communication In Marlmentioning

confidence: 99%

AB-Mapper: Attention and BicNet Based Multi-agent Path Finding for Dynamic Crowded Environment

Guan,

Gao,

Zhao

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-agent path finding in dynamic crowded environments is of great academic and practical value for multirobot systems in the real world. To improve the effectiveness and efficiency of communication and learning process during path planning in dynamic crowded environments, we introduce an algorithm called Attention and BicNet based Multiagent path planning with effective reinforcement (AB-Mapper) under the actor-critic reinforcement learning framework. In this framework, on the one hand, we utilize the BicNet with communication function in the actor-network to achieve intra team coordination. On the other hand, we propose a centralized critic network that can selectively allocate attention weights to surrounding agents. This attention mechanism allows an individual agent to automatically learn a better evaluation of actions by also considering the behaviours of its surrounding agents. Compared with the state-of-the-art method Mapper, our AB-Mapper is more effective (85.86% vs. 81.56% in terms of success rate) in solving the general path finding problems with dynamic obstacles. In addition, in crowded scenarios, our method outperforms the Mapper method by a large margin, reaching a stunning gap of more than 40% for each experiment.

show abstract

Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

Cited by 21 publications

References 18 publications

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Self-Organized Polynomial-Time Coordination Graphs

AB-Mapper: Attention and BicNet Based Multi-agent Path Finding for Dynamic Crowded Environment

Contact Info

Product

Resources

About