Google Research Football: A Novel Reinforcement Learning Environment

Kurach, Karol; Raichuk, Anton; Stańczyk, Piotr; Zając, Michał; Bachem, Olivier; Espeholt, Lasse; Riquelme, Carlos; Vincent, Damien; Michalski, Marcin; Bousquet, Olivier; Gelly, Sylvain

doi:10.48550/arxiv.1907.11180

Cited by 19 publications

(28 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we will examine whether our pre-trained offline RL models can accelerate the multi-agent reinforcement learning training process. Thus, we consider five GRF academic scenarios originally proposed in Kurach et al (2019), including:…”

Section: Accelerate Multi-agent Reinforcement Learning Trainingmentioning

confidence: 99%

“…Deep reinforcement learning (DRL) has shown great success in many video games, including the Atari games (Mnih et al, 2013), StarCraft II (Vinyals et al, 2019), Dota II (Berner et al, 2019), etc. However, current DRL systems still suffer from challenges of multi-agent coordination (Rashid et al, 2018;Mahajan et al, 2019;Yu et al, 2021), sparse rewards (Taiga et al, 2019;Zhang et al, 2020), stochastic environments (Kurach et al, 2019;Team et al, 2021), etc. In seeking to address these challenges, we employ a football video game, e.g., Google Research Football (GRF) (Kurach et al, 2019), as our testbed.…”

Section: Introductionmentioning

confidence: 99%

“…Even much work has been done recently (Kurach et al, 2019;Li et al, 2021;Liu et al, 2021b), there remain many problems in building agents for the GRF: (1) Multiple Players: In the GRF, there are both cooperative and competitive players. For cooperative players, the joint action space is very huge, thus it is hard to build a single agent to control all the players.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Huang

Chen

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has achieved super-human performance on complex video games (e.g., StarCraft II and Dota II). However, current DRL systems still suffer from challenges of multi-agent coordination, sparse rewards, stochastic environments, etc. In seeking to address these challenges, we employ a football video game, e.g., Google Research Football (GRF), as our testbed and develop an end-to-end learning-based AI system (denoted as TiKick 23 ) to complete this challenging task. In this work, we first generated a large replay dataset from the self-playing of single-agent experts, which are obtained from league training. We then developed a distributed learning system and new offline algorithms to learn a powerful multi-agent AI from the fixed single-agent dataset. To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game, while previous work could either control a single agent or experiment on toy academic scenarios. Extensive experiments further show that our pre-trained model can accelerate the training process of the modern multi-agent algorithm and our method achieves state-of-theart performances on various academic scenarios. * Equal contribution 2 Codes can be found at https://github.com/TARTRL/TiKick.3 Videos available at https://sites.google.com/view/tikick.Preprint. Under review.

show abstract

Section: Accelerate Multi-agent Reinforcement Learning Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Huang

Chen

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We consider two multi-agent domains, Google Research Football (Kurach et al, 2019) (GRF) and DeepMind Lab 2D (Beattie et al, 2020) (DMLab2D), with a focus on the more dynamically complex GRF.…”

Section: Environments and Setupsmentioning

confidence: 99%

Agent-Centric Representations for Multi-Agent Reinforcement Learning

Shang,

Espeholt,

Raichuk

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Object-centric representations have recently enabled significant progress in tackling relational reasoning tasks. By building a strong object-centric inductive bias into neural architectures, recent efforts have improved generalization and data efficiency of machine learning algorithms for these problems. One problem class involving relational reasoning that still remains under-explored is multi-agent reinforcement learning (MARL).Here we investigate whether object-centric representations are also beneficial in the fully cooperative MARL setting. Specifically, we study two ways of incorporating an agent-centric inductive bias into our RL algorithm: 1. Introducing an agent-centric attention module with explicit connections across agents 2. Adding an agentcentric unsupervised predictive objective (i.e. not using action labels), to be used as an auxiliary loss for MARL, or as the basis of a pre-training step. We evaluate these approaches on the Google Research Football environment as well as DeepMind Lab 2D. Empirically, agent-centric representation learning leads to the emergence of more complex cooperation strategies between agents as well as enhanced sample efficiency and generalization.

show abstract

“…An obvious application for generalized pixel-based representations is game playing agents. For example, the current learned representation can be used with the Google Research Football Environment [30] to initialize the visual encoder of an imitation-or reinforcement-learning agent. With most genre-specific information (such as pattern of football pitch, goalposts) already present in these representations, fine-tuning the task-specific visual information (such as position of players, ball) or learning the control-policy becomes much more sample efficient compared to starting from scratch.…”

Section: Applications Of General Representationsmentioning

confidence: 99%

Contrastive Learning of Generalized Game Representations

Trivedi¹,

Liapis²,

Yannakakis³

2021

Preprint

View full text Add to dashboard Cite

Representing games through their pixels offers a promising approach for building general-purpose and versatile game models. While games are not merely images, neural network models trained on game pixels often capture differences of the visual style of the image rather than the content of the game. As a result, such models cannot generalize well even within similar games of the same genre. In this paper we build on recent advances in contrastive learning and showcase its benefits for representation learning in games. Learning to contrast images of games not only classifies games in a more efficient manner; it also yields models that separate games in a more meaningful fashion by ignoring the visual style and focusing, instead, on their content. Our results in a large dataset of sports video games containing 100k images across 175 games and 10 game genres suggest that contrastive learning is better suited for learning generalized game representations compared to conventional supervised learning. The findings of this study bring us closer to universal visual encoders for games that can be reused across previously unseen games without requiring retraining or fine-tuning.Index Terms-computer vision in games, generalized representations, contrastive learning This project has received funding from the EU's Horizon 2020 programme under grant agreement No 951911.

show abstract

Google Research Football: A Novel Reinforcement Learning Environment

Cited by 19 publications

References 0 publications

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Agent-Centric Representations for Multi-Agent Reinforcement Learning

Contrastive Learning of Generalized Game Representations

Contact Info

Product

Resources

About