Heterogeneous Graph Attention Networks for Learning Diverse Communication

Abstract

Multi-agent teaming achieves better performance when there is communication among participating agents allowing them to coordinate their actions for maximizing shared utility. However, when collaborating a team of agents with different action and observation spaces, information sharing is not straightforward and requires customized communication protocols, depending on sender and receiver types. Without properly modeling such heterogeneity in agents, communication becomes less helpful and could even deteriorate the multi-agent cooperation performance. We propose heterogeneous graph attention networks, called HetNet, to learn efficient and diverse communication models for coordinating heterogeneous agents towards accomplishing tasks that are of collaborative nature. We propose a Multi-Agent Heterogeneous Actor-Critic (MAHAC) learning paradigm to obtain collaborative per-class policies and effective communication protocols for composite robot teams. Our proposed framework is evaluated against multiple baselines in a complex environment in which agents of different types must communicate and cooperate to satisfy the objectives. Experimental results show that HetNet outperforms the baselines in learning sophisticated multi-agent communication protocols by achieving ∼10% improvements in performance metrics.

Heterogeneous Multi-Agent Communication Learning via Graph Information Maximization

“…In this section, we select Predator-Capture-Prey (PCP) [16] and StarCraft II Multi-Agent Challenge (SMAC) [18] as our benchmarks. We conduct various experiments on these benchmarks with GPU Nvidia RTX 2080 to answer: Q1: Whether CLAR can improve performance in diverse heterogeneous scenarios?…”

“…However, most of these methods are not specially designed for heterogeneous scenarios, where agents have heterogeneous attributes or features based on different observation spaces or action sets. Although some works attempt to use heterogeneous GNNs to learn communication in heterogeneous scenarios [15], [16], these works do not further optimize the high-level message representation, resulting in communication learning less effective. Different from them, the proposed method utilizes MI optimization to obtain highquality message representations for action selection.…”

“…In our work, we formulate the heterogeneous multi-agent issue as the Heterogeneous Multi-Agent POMDP represented by G = ⟨C, I, {S c } c∈C , {A c } c∈C , {O c } c∈C , R⟩ [16]. C represents the set of all classes of agents in the heterogeneous scenarios and the index c ∈ C represents the class that the agent belongs to, the total number of classes is denoted as i ∈ O c and takes an action a c i ∈ A c , which forms the joint action a ∈ {A c } c∈C .…”

Multi-UAV Planning for Cooperative Wildfire Coverage and Tracking with Quality-of-Service Guarantees

“…Ure et al [85] proposed a decentralized approach for the problem of multiple learning and collaborating agents in fire monitoring cases where agents estimate different models from their local observations, but they can share information by communicating model parameters. In [86,87], a graph-based actor-critic [84] method is introduced to learn efficient communication protocols for a group of cooperating heterogeneous agents (i.e., sensing and manipulating agents) performing wildfire fighting.…”

Stochastic Resource Optimization over Heterogeneous Graph Neural Networks for Failure-Predictive Maintenance Scheduling

“…Graph Attention Networks (GATs) (Veličković et al 2018) were proposed to learn the importance between nodes and its neighbors and fuse the neighbors by normalized attention coefficients. Recently, researchers have proposed heterogeneous GNNs, allowing for learning with different types of nodes and edges, showing superior performance and model expressiveness (Wang et al 2019;Seraj et al 2021).…”