A review of cooperative multi-agent deep reinforcement learning

Oroojlooyjadid, Afshin; Hajinezhad, Davood

doi:10.1007/s10489-022-04105-y

Cited by 102 publications

(36 citation statements)

References 125 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Zhang et al (2021), a selective overview of theories and algorithms on multi-agent reinforcement learning is presented. In Oroojlooy and Hajinezhad (2022), a review on cooperative multi-agent deep reinforcement learning is given.…”

Section: Reinforcement Learning In Multi-agent Search Tasksmentioning

confidence: 99%

On the role and opportunities in teamwork design for advanced multi-robot search systems

Francos

Bruckstein²

2023

Front. Robot. AI

View full text Add to dashboard Cite

Intelligent robotic systems are becoming ever more present in our lives across a multitude of domains such as industry, transportation, agriculture, security, healthcare and even education. Such systems enable humans to focus on the interesting and sophisticated tasks while robots accomplish tasks that are either too tedious, routine or potentially dangerous for humans to do. Recent advances in perception technologies and accompanying hardware, mainly attributed to rapid advancements in the deep-learning ecosystem, enable the deployment of robotic systems equipped with onboard sensors as well as the computational power to perform autonomous reasoning and decision making online. While there has been significant progress in expanding the capabilities of single and multi-robot systems during the last decades across a multitude of domains and applications, there are still many promising areas for research that can advance the state of cooperative searching systems that employ multiple robots. In this article, several prospective avenues of research in teamwork cooperation with considerable potential for advancement of multi-robot search systems will be visited and discussed. In previous works we have shown that multi-agent search tasks can greatly benefit from intelligent cooperation between team members and can achieve performance close to the theoretical optimum. The techniques applied can be used in a variety of domains including planning against adversarial opponents, control of forest fires and coordinating search-and-rescue missions. The state-of-the-art on methods of multi-robot search across several selected domains of application is explained, highlighting the pros and cons of each method, providing an up-to-date view on the current state of the domains and their future challenges.

show abstract

Section: Reinforcement Learning In Multi-agent Search Tasksmentioning

confidence: 99%

On the role and opportunities in teamwork design for advanced multi-robot search systems

Francos

Bruckstein²

2023

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…This focus has clearly been missing in the Deep RL field. While multi-agent reinforcement learning (MARL) is a well-established branch of Deep RL, most learning algorithms and environments proposed have targeted a relatively small number of agents (Foerster et al 2016; OroojlooyJadid and Hajinezhad, 2019), and thus not sufficient to study the emergent properties from large populations. In the most common MARL environments (Resnick et al 2018; Baker et al 2019; Jaderberg et al 2019; Terry et al 2020), “multi-agent” simply means two or four agent trained to perform a task by means of self-play (Bansal et al 2017; Liu et al 2019; Ha, 2020).…”

Section: Collective Intelligence For Deep Learningmentioning

confidence: 99%

Collective intelligence for deep learning: A survey of recent developments

Tang

2022

Collective Intelligence

View full text Add to dashboard Cite

In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled practitioners to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Collective behavior, commonly observed in nature, tends to produce systems that are robust, adaptable, and have less rigid assumptions about the environment configuration. Collective intelligence, as a field, studies the group intelligence that emerges from the interactions of many individuals. Within this field, ideas such as self-organization, emergent behavior, swarm optimization, and cellular automata were developed to model and explain complex systems. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research’s involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its capabilities. We hope this review can serve as a bridge between the complex systems and deep learning communities.

show abstract

“…The HetGAT Enc-Dec policy achieved the highest fleet rewards, followed by HetGAT and HetGCN. For a thorough discussion on CTDE MARL, we refer our readers to [37]. For the HetGAT policy, we only used a module resembling encoder architecture, however, with scalar outputs for the depot representations h d , used as the actionvalue function outputs.…”

Section: B One-shot Trainingmentioning

confidence: 99%

CoCo Games: Graphical Game-Theoretic Swarm Control for Communication-Aware Coverage

Fernando

Senanayake

Swany

2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Autonomous mobility is emerging as a new mode of urban transportation for moving cargo and passengers. However, such fleet coordination schemes face significant challenges in scaling to accommodate fast-growing fleet sizes that vary in their operational range, capacity, and communication capabilities. We introduce the concept of partially observable advanced air mobility games to coordinate a fleet of aerial vehicle agents accounting for their heterogeneity and self-interest inherent to commercial mobility fleets. We propose a novel heterogeneous graph attention-based encoder-decoder (HetGAT Enc-Dec) neural network to construct a generalizable stochastic policy stemming from the inter-and intra-agent relations within the mobility system. We train our policy by leveraging deep multiagent reinforcement learning, allowing decentralized decisionmaking for the agents using their local observations. Through extensive experimentation, we show that the fleets operating under the HetGAT Enc-Dec policy outperform other state-of-theart graph neural network-based policies by achieving the highest fleet reward and fulfillment ratios in an on-demand mobility network.

show abstract

A review of cooperative multi-agent deep reinforcement learning

Cited by 102 publications

References 125 publications

On the role and opportunities in teamwork design for advanced multi-robot search systems

On the role and opportunities in teamwork design for advanced multi-robot search systems

Collective intelligence for deep learning: A survey of recent developments

CoCo Games: Graphical Game-Theoretic Swarm Control for Communication-Aware Coverage

Contact Info

Product

Resources

About