A Review of Cooperative Multi-Agent Deep Reinforcement Learning

Oroojlooyjadid, Afshin; Hajinezhad, Davood

doi:10.48550/arxiv.1908.03963

Cited by 58 publications

(66 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As a result, distributed training and execution are more desired for multi-agent adversary systems. We refer the readers to [9,24] for a detailed discussion about MARL.…”

Section: Multi-agent Rl (Marl)mentioning

confidence: 99%

Multi-Agent Adversarial Attacks for Multi-Channel Communications

Dong¹,

Wu²,

Sultani³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently Reinforcement Learning (RL) has been applied as an antiadversarial remedy in wireless communication networks. However studying the RL-based approaches from the adversary's perspective has received little attention. Additionally, RL-based approaches in an anti-adversary or adversarial paradigm mostly consider singlechannel communication (either channel selection or single channel power control), while multi-channel communication is more common in practice. In this paper, we propose a multi-agent adversary system (MAAS) for modeling and analyzing adversaries in a wireless communication scenario by careful design of the reward function under realistic communication scenarios. In particular, by modeling the adversaries as learning agents, we show that the proposed MAAS is able to successfully choose the transmitted channel(s) and their respective allocated power(s) without any prior knowledge of the sender strategy. Compared to the single-agent adversary (SAA), multi-agents in MAAS can achieve significant reduction in signal-to-noise ratio (SINR) under the same power constraints and partial observability, while providing improved stability and a more efficient learning process. Moreover, through empirical studies we show that the results in simulation are close to the ones in communication in reality, a conclusion that is pivotal to the validity of performance of agents evaluated in simulations.

show abstract

“…As a result, distributed training and execution are more desired for multi-agent adversary systems. We refer the readers to [9,24] for a detailed discussion about MARL.…”

Section: Multi-agent Rl (Marl)mentioning

confidence: 99%

Multi-Agent Adversarial Attacks for Multi-Channel Communications

Dong¹,

Wu²,

Sultani³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While multi-agent reinforcement learning (MARL) is a well-established branch of Deep RL, most learning algorithms and environments proposed have targeted a relatively small number of agents 17,41 . It is common to see environments with less dozens of agents 1,30,52,67 , with 2-agent and 4-agent environments being particular popular for the study of competitive, self-play settings 2,23,36 .…”

Section: Multi-agent Learningmentioning

confidence: 99%

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Ha¹,

Tang²

2021

Preprint

View full text Add to dashboard Cite

In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled researchers and practitioners alike to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Ideas from collective intelligence, in particular concepts from complex systems such as self-organization, emergent behavior, swarm optimization, and cellular systems tend to produce solutions that are robust, adaptable, and have less rigid assumptions about the environment configuration. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research's involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its current capabilities. To facilitate a bi-directional flow of ideas, we also discuss work that utilize modern deep learning models to help advance complex systems research. We hope this review can serve as a bridge between complex systems and deep learning communities to facilitate the cross pollination of ideas and foster new collaborations across disciplines.

show abstract

“…Multi-agent reinforcement learning (OroojlooyJadid & Hajinezhad, 2019) is challenged by the size of joint action space, which grows exponentially with the number of agents. Independent Q-learning (Tan, 1993;Foerster et al, 2017) models agents as independent learners, which makes the environment non-stationary in the perspective of each agent.…”

Section: Related Workmentioning

confidence: 99%

Self-Organized Polynomial-Time Coordination Graphs

Yang¹,

Dong²,

Ren³

et al. 2021

Preprint

View full text Add to dashboard Cite

Coordination graph is a promising approach to model agent collaboration in multiagent reinforcement learning. It factorizes a large multi-agent system into a suite of overlapping groups that represent the underlying coordination dependencies. One critical challenge in this paradigm is the complexity of computing maximumvalue actions for a graph-based value factorization. It refers to the decentralized constraint optimization problem (DCOP), which and whose constant-ratio approximation are NP-hard problems. To bypass this fundamental hardness, this paper proposes a novel method, named Self-Organized Polynomial-time Coordination Graphs (SOP-CG), which uses structured graph classes to guarantee the optimality of the induced DCOPs with sufficient function expressiveness. We extend the graph topology to be state-dependent, formulate the graph selection as an imaginary agent, and finally derive an end-to-end learning paradigm from the unified Bellman optimality equation. In experiments, we show that our approach learns interpretable graph topologies, induces effective coordination, and improves performance across a variety of cooperative multi-agent tasks.

show abstract

A Review of Cooperative Multi-Agent Deep Reinforcement Learning

Cited by 58 publications

References 84 publications

Multi-Agent Adversarial Attacks for Multi-Channel Communications

Multi-Agent Adversarial Attacks for Multi-Channel Communications

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Self-Organized Polynomial-Time Coordination Graphs

Contact Info

Product

Resources

About