Multiplayer Support for the Arcade Learning Environment

Terry, Justin K.; Black, Benjamin

doi:10.48550/arxiv.2009.09341

Cited by 4 publications

(2 citation statements)

References 11 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiments were performed on multiple multi-agent environments from the PettingZoo library (Terry et al, 2020b ), which contains the Multi-Agent Particle Environments (MPE) (Lowe et al, 2017 ; Mordatch and Abbeel, 2017 ) and multi-agent variants of the Atari 2600 Arcade Learning Environment (ALE) (Bellemare et al, 2013 ; Terry and Black, 2020 ).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Investigation of independent reinforcement learning algorithms in multi-agent environments

Lee

Subramanian

Crowley

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on seven PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. For the cooperative setting, we show that independent algorithms can perform on par with multi-agent algorithms in fully-observable environments, while adding recurrence improves the learning of independent algorithms in partially-observable environments. In the competitive setting, independent algorithms can perform on par or better than multi-agent algorithms, even in more challenging environments. We also show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies in mixed environments.

show abstract

Section: Methodsmentioning

confidence: 99%

“…For the cooperative setting, experiments were performed on a modified version of the 2-player Space Invaders (Bellemare et al, 2013 ; Terry and Black, 2020 ), and the Simple Reference MPE environment (Lowe et al, 2017 ; Mordatch and Abbeel, 2017 ). In Space Invaders, both agents share the common goal of shooting down all aliens.…”

Section: Methodsmentioning

confidence: 99%

Investigation of independent reinforcement learning algorithms in multi-agent environments

Lee

Subramanian

Crowley

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

PettingZoo: Gym for Multi-Agent Reinforcement Learning

Terry¹,

Black²,

Grammel

et al. 2020

Preprint

View full text Add to dashboard Cite

OpenAI's Gym library contains a large, diverse set of environments that are useful benchmarks in reinforcement learning, under a single elegant Python API (with tools to develop new compliant environments) . The introduction of this library has proven a watershed moment for the reinforcement learning community, because it created an accessible set of benchmark environments that everyone could use (including wrapper important existing libraries), and because a standardized API let RL learning methods and environments from anywhere be trivially exchanged. This paper similarly introduces PettingZoo, a library of diverse set of multi-agent environments under a single elegant Python API, with tools to easily make new compliant environments.

show abstract

A coevolutionary approach to deep multi-agent reinforcement learning

Klijn

Eiben

2021

Preprint

View full text Add to dashboard Cite

Traditionally, Deep Artificial Neural Networks (DNN's) are trained through gradient descent. Recent research shows that Deep Neuroevolution (DNE) is also capable of evolving multi-million-parameter DNN's, which proved to be particularly useful in the field of Reinforcement Learning (RL). This is mainly due to its excellent scalability and simplicity compared to the traditional MDP-based RL methods. So far, DNE has only been applied to complex singleagent problems. As evolutionary methods are a natural choice for multi-agent problems, the question arises whether DNE can also be applied in a complex multi-agent setting. In this paper, we describe and validate a new approach based on Coevolution. To validate our approach, we benchmark two Deep Coevolutionary Algorithms on a range of multi-agent Atari games and compare our results against the results of Ape-X DQN. Our results show that these Deep Coevolutionary algorithms (1) can be successfully trained to play various games, (2) outperform Ape-X DQN in some of them, and therefore (3) show that Coevolution can be a viable approach to solving complex multi-agent decision-making problems.

show abstract

Multiplayer Support for the Arcade Learning Environment

Cited by 4 publications

References 11 publications

Investigation of independent reinforcement learning algorithms in multi-agent environments

Investigation of independent reinforcement learning algorithms in multi-agent environments

PettingZoo: Gym for Multi-Agent Reinforcement Learning

A coevolutionary approach to deep multi-agent reinforcement learning

Contact Info

Product

Resources

About