CoCo Games: Graphical Game-Theoretic Swarm Control for Communication-Aware Coverage

Abstract

Autonomous mobility is emerging as a new mode of urban transportation for moving cargo and passengers. However, such fleet coordination schemes face significant challenges in scaling to accommodate fast-growing fleet sizes that vary in their operational range, capacity, and communication capabilities. We introduce the concept of partially observable advanced air mobility games to coordinate a fleet of aerial vehicle agents accounting for their heterogeneity and self-interest inherent to commercial mobility fleets. We propose a novel heterogeneous graph attention-based encoder-decoder (HetGAT Enc-Dec) neural network to construct a generalizable stochastic policy stemming from the inter-and intra-agent relations within the mobility system. We train our policy by leveraging deep multiagent reinforcement learning, allowing decentralized decisionmaking for the agents using their local observations. Through extensive experimentation, we show that the fleets operating under the HetGAT Enc-Dec policy outperform other state-of-theart graph neural network-based policies by achieving the highest fleet reward and fulfillment ratios in an on-demand mobility network.