Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

Rudin, Nikita; Hoeller, David; Reist, Philipp; Hutter, Marco

doi:10.48550/arxiv.2109.11978

Cited by 16 publications

(30 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collective intelligence observed in nature, however, rely on a much larger number of individuals than typically studied in MARL, involving population sizes from thousands to million. Recent advances in Deep RL have demonstrated the capabilities of simulating thousands of agents in complex 3D simulation environments using only a single GPU 25,53 . A key challenge is in approaching the problem of multi-agent learning at a much larger scale, leveraging such advances in parallel computing hardware and distributed computation, with the goal of training millions of agents.…”

Section: Multi-agent Learningmentioning

confidence: 99%

“…Researchers and practitioners alike soon quickly incorporated DL to address the long-standing problems in several other fields spanning computer vision (CV) 24,47,60 , natural language processing (NLP) 5,48,49 , reinforcement learning (RL) 37,59,68 and computational biology 32,58 , many of which have technological breakthroughs and achieved state-of-the-art results. 53 . Such advances opens the door for large scale 3D simulation of artificial agents that can interact with each other and collectively develop intelligent behavior.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, by modeling each sensory neuron as an individual neural network and allowing communications between each other in a layer, a model having this layer is able to handle randomly shuffled inputs of arbitrary sizes 65 . Finally, Figure 1 shows the results from training thousands of RL agents in simulation simultaneously on a single GPU 53 . With continuous technological advances in hardware-acceleration, we can certainly expect to see more works that incorporate swarm computing concepts in population-based deep RL.…”

Section: Introductionmentioning

confidence: 99%

“…Figure 1. Recent advances in GPU hardware enables realistic 3D simulation of thousands of robot models, such as the one shown in this figure from Rudin et al53 . Such advances opens the door for large scale 3D simulation of artificial agents that can interact with each other and collectively develop intelligent behavior.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Ha¹,

Tang²

2021

Preprint

View full text Add to dashboard Cite

In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled researchers and practitioners alike to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Ideas from collective intelligence, in particular concepts from complex systems such as self-organization, emergent behavior, swarm optimization, and cellular systems tend to produce solutions that are robust, adaptable, and have less rigid assumptions about the environment configuration. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research's involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its current capabilities. To facilitate a bi-directional flow of ideas, we also discuss work that utilize modern deep learning models to help advance complex systems research. We hope this review can serve as a bridge between complex systems and deep learning communities to facilitate the cross pollination of ideas and foster new collaborations across disciplines.

show abstract

Section: Multi-agent Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Ha¹,

Tang²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The behaviors were learned using a curriculum, highly task specific rewards and feature observations based on step-bystep target foothold locations. Other types of curricula have been used together with shaping rewards to learn blind [1] and very recently also depth perceptive [16] terrain adaptive controllers via RL. It is not clear whether such curricula can easily be designed for any type of terrain.…”

Section: Related Workmentioning

confidence: 99%

Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

Brakel¹,

Bohez²,

Hasenclever³

et al. 2021

Preprint

View full text Add to dashboard Cite

Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear solvers can produce coordinated constraint satisfying motions, but often take too long to converge for online application. RL methods can learn dynamic reactive controllers but require carefully tuned shaping rewards to produce good gaits and can have trouble discovering precise coordinated movements. Imitation learning circumvents this problem and has been used with motion capture data to extract quadruped gaits for flat terrains. However, it would be costly to acquire motion capture data for a very large variety of terrains with height differences. In this work, we combine the advantages of trajectory optimization and learning methods and show that terrain adaptive controllers can be obtained by training policies to imitate trajectories that have been planned over procedural terrains by a non-linear solver. We show that the learned policies transfer to unseen terrains and can be fine-tuned to dynamically traverse challenging terrains that require precise foot placements and are very hard to solve with standard RL.

show abstract

ART: Avatar Robotics Telepresence—the future of humanoid material handling loco-manipulation

Vaz,

Kosanovic,

2023

Intel Serv Robotics

View full text Add to dashboard Cite

Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

Cited by 16 publications

References 0 publications

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

ART: Avatar Robotics Telepresence—the future of humanoid material handling loco-manipulation

Contact Info

Product

Resources

About