Open-Ended Learning Leads to Generally Capable Agents

Team, Open Ended Learning; Stooke, Adam; Mahajan, Anuj; Barros, Catarina; Deck, Charlie; Bauer, Jakob; Sygnowski, Jakub; Trebacz, Maja; Jaderberg, Max; Mathieu, Michaël; McAleese, Nat; Nathalie, Bradley-Schmieg,; Wong, Nathaniel; Porcel, Nicolas; Raileanu, Roberta; Hughes-Fitt, Steph; Dalibard, Valentin; Czarnecki, Wojciech Marian

doi:10.48550/arxiv.2107.12808

Cited by 8 publications

(13 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Like human children, DL agents might need to be and learn in an open-ended environment, where ToM skills are necessary and might be acquired through interaction with other agents. Recent work [45][46][47][48][49] has shown how powerful open-endedness can be for learning complex behavior. Particularly, [48] introduces XLand, a vast environment where multiple agents learn from a spectrum of completely cooperative to fully competitive tasks.…”

Section: Shortcuts In Theory Of Mind Tasksmentioning

confidence: 99%

“…Recent work [45][46][47][48][49] has shown how powerful open-endedness can be for learning complex behavior. Particularly, [48] introduces XLand, a vast environment where multiple agents learn from a spectrum of completely cooperative to fully competitive tasks. Agents trained on XLand learn complex strategies to solve any given task, but it is unknown whether ToM is one of these strategies.…”

Section: Shortcuts In Theory Of Mind Tasksmentioning

confidence: 99%

“…Applying similar studies to agents trained in cooperative and/or competitive environments like XLand [48], Hide and Seek [26] or Capture the Flag [92] could reveal ToM-like capabilities present in these agents. Moreover, these approaches may show what pressures in the environment are important for ToM to emerge.…”

Section: Evaluating Theory Of Mind In DL Agentsmentioning

confidence: 99%

See 2 more Smart Citations

Mind the gap: Challenges of deep learning approaches to Theory of Mind

Aru¹,

Labash²,

Corcoll³

et al. 2022

Preprint

View full text Add to dashboard Cite

Theory of Mind is an essential ability of humans to infer the mental states of others. Here we provide a coherent summary of the potential, current progress, and problems of deep learning approaches to Theory of Mind. We highlight that many current findings can be explained through shortcuts. These shortcuts arise because the tasks used to investigate Theory of Mind in deep learning systems have been too narrow. Thus, we encourage researchers to investigate Theory of Mind in complex openended environments. Furthermore, to inspire future deep learning systems we provide a concise overview of prior work done in humans. We further argue that when studying Theory of Mind with deep learning, the research's main focus and contribution ought to be opening up the network's representations. We recommend researchers to use tools from the field of interpretability of AI to study the relationship between different network components and aspects of Theory of Mind.

show abstract

Section: Shortcuts In Theory Of Mind Tasksmentioning

confidence: 99%

Section: Shortcuts In Theory Of Mind Tasksmentioning

confidence: 99%

See 1 more Smart Citation

Mind the gap: Challenges of deep learning approaches to Theory of Mind

Aru¹,

Labash²,

Corcoll³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Open-ended learning [56,13,16,57,18,51] is closely related to unsupervised skill discovery. However, most approaches require either a parameterizable environment [56,57], some fixed encoding of tasks [51] or self-competition [47,5]. This limits the applicability to environments that are engineered with these restrictions in mind.…”

Section: Related Workmentioning

confidence: 99%

Open-Ended Reinforcement Learning with Neural Reward Functions

Meier¹,

Mujika²

2022

Preprint

View full text Add to dashboard Cite

Inspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more on unsupervised discovery of skills. Most current approaches, like DIAYN or DADS, optimize some form of mutual information objective. We propose a different approach that uses reward functions encoded by neural networks. These are trained iteratively to reward more complex behavior. In high-dimensional robotic environments our approach learns a wide range of interesting skills including front-flips for HALF-CHEETAH and onelegged running for HUMANOID. In the pixel-based Montezuma's Revenge environment our method also works with minimal changes and it learns complex skills that involve interacting with items and visiting diverse locations. A web version of this paper which shows animations for the different skills is available in https://as.inf.ethz.ch/research/open_ended_RL/main.html * Equal contribution.

show abstract

“…Understanding how such an apparently simple optimization procedure, based on variation and selection, can generate this impressive diversity of species varying in their morphological, behavioral or cultural repertoiresm has been a puzzle for several research communities -including evolutionary biology, artificial life (AL) and artificial intelligence (AI). An important driver of recent progress in our understanding of the emergence of open-endedness in artificial systems is that tasks and environments play a key role, complementary to that of the cognitive mechanisms that the AI community has been focusing on for decades [36]. Firm steps towards environments of increased complexity were taken under a family of techniques termed as autocurricula [31], where aspects of an artificial system such as its multi-agent dynamics [2,23] and curriculum learning [39] are leveraged to automate the emergence of complexity rather than hand-engineer it.…”

Section: Introductionmentioning

confidence: 99%

Plasticity and evolvability under environmental variability: the joint role of fitness-based selection and niche-limited competition

Nisioti¹,

Moulin-Frier²

2022

Preprint

View full text Add to dashboard Cite

The diversity and quality of natural systems has been a puzzle and inspiration for communities studying artificial life. It is now widely admitted that the adaptation mechanisms enabling these properties are largely influenced by the environments they inhabit. Organisms facing environmental variability have two alternative adaptation mechanisms operating at different timescales: plasticity, the ability of a phenotype to survive in diverse environments and evolvability, the ability to adapt through mutations. Although vital under environmental variability, both mechanisms are associated with fitness costs hypothesized to render them unnecessary in stable environments. In this work, we study the interplay between environmental dynamics and adaptation in a minimal model of the evolution of plasticity and evolvability. We experiment with different types of environments characterized by the presence of niches and a climate function that determines the fitness landscape. We empirically show that environmental dynamics affect plasticity and evolvability differently and that the presence of diverse ecological niches favors adaptability even in stable environments. We perform ablation studies of the selection mechanisms to separate the role of fitness-based selection and niche-limited competition. Results obtained from our minimal model allow us to propose promising research directions in the study of open-endedness in biological and artificial systems.

show abstract

Open-Ended Learning Leads to Generally Capable Agents

Cited by 8 publications

References 32 publications

Mind the gap: Challenges of deep learning approaches to Theory of Mind

Mind the gap: Challenges of deep learning approaches to Theory of Mind

Open-Ended Reinforcement Learning with Neural Reward Functions

Plasticity and evolvability under environmental variability: the joint role of fitness-based selection and niche-limited competition

Contact Info

Product

Resources

About