rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch

Stooke, Adam; Abbeel, Pieter

doi:10.48550/arxiv.1909.01500

Cited by 34 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For all approaches, including baselines, we run at least 15 random seeds for 6 different learning rates, {0.001, 0.003, 0.0001, 0.0003, 0.00001, 0.00003}, and report the best learning rate for each. Other hyperparameters are taken as default in the codebase (Stooke & Abbeel, 2019;van der Pol et al, 2020).…”

Section: Methodsmentioning

confidence: 99%

Multi-Agent MDP Homomorphic Networks

Pol¹,

Hoof²,

Oliehoek³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper introduces Multi-Agent MDP Homomorphic Networks, a class of networks that allows distributed execution using only local information, yet is able to share experience between global symmetries in the joint state-action space of cooperative multi-agent systems. In cooperative multi-agent systems, complex symmetries arise between different configurations of the agents and their local observations. For example, consider a group of agents navigating: rotating the state globally results in a permutation of the optimal joint policy. Existing work on symmetries in single agent reinforcement learning can only be generalized to the fully centralized setting, because such approaches rely on the global symmetry in the full state-action spaces, and these can result in correspondences across agents. To encode such symmetries while still allowing distributed execution we propose a factorization that decomposes global symmetries into local transformations. Our proposed factorization allows for distributing the computation that enforces global symmetries over local agents and local interactions. We introduce a multi-agent equivariant policy network based on this factorization. We show empirically on symmetric multi-agent problems that distributed execution of globally symmetric policies improves data efficiency compared to non-equivariant baselines.

show abstract

Section: Methodsmentioning

confidence: 99%

Multi-Agent MDP Homomorphic Networks

Pol¹,

Hoof²,

Oliehoek³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While H2.0 is a fast simulator, we find that the performance of the overall simulation+training loop is bottlenecked by factors like synchronization of parallel environments and reloading of assets upon episode reset. An exciting and complementary future direction is holistically reorganizing the rendering+physics+RL interplay as studied by [75][76][77][78][79][80]. Concretely, as illustrated in Figure 3, there is idle GPU time when rendering is faster than physics, because inference waits for both o t and s t+1 to be ready despite not needing s t+1 .…”

Section: Societal Impacts Limitations and Conclusionmentioning

confidence: 99%

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Szot¹,

Clegg²,

Undersander³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack -data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850× real-time) on an 8-GPU node, representing 100× speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, prepare groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from 'hand-off problems', and (3) SPA pipelines are more brittle than RL policies. Figure 1: A mobile manipulator (Fetch robot) simulated in Habitat 2.0 performing rearrangement tasks in a ReplicaCAD apartment -(left) opening a drawer before picking up an item from it, and (right) placing an object into the bowl after navigating to the table. Best viewed in motion at https://sites.google.com/view/habitat2. Preprint. Under review.

show abstract

“…• rlpyt is an open-source repository of modular and parallelised implementations of various deep reinforcement-learning algorithms [569].…”

Section: Literature Walkthroughmentioning

confidence: 99%

Social physics

Jusup,

Holme,

Kanazawa

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent decades have seen a rise in the use of physics-inspired or physicslike methods in attempts to resolve diverse societal problems. Such a rise is driven both by physicists venturing outside of their traditional domain of interest, but also by scientists from other domains who wish to mimic the enormous success of physics throughout the 19 th and 20 th century. Here, we dub the physics-inspired and physics-like work on societal problems "social physics", and pay our respect to intellectual mavericks who nurtured the field to its maturity. We do so by comprehensively (but not exhaustively) reviewing the current state of the art. Starting with a set of topics that pertain to the modern way of living and factors that enable humankind's prosperous existence, we discuss urban development and traffic, the functioning of financial markets, cooperation as a basis for civilised life, the structure of (social) networks, and the integration of intelligent machines in such networks. We then shift focus to a set of topics that explore potential threats to humanity. These include criminal behaviour, massive migrations, contagions, environmental problems, and finally climate change. The coverage of each topic is ended with ideas for future progress. Based on the number of ideas laid out, but also on the fact that the field is already too big for an exhaustive review despite our best efforts, we are forced to conclude that the future for social physics is bright. Physicists tackling societal problems are no longer a curiosity, but rather a force to be reckoned with, yet for reckoning to be truly productive, it is necessary to build dialog and mutual understanding with social scientists, environmental scientists, philosophers, and more.

show abstract

rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch

Cited by 34 publications

References 15 publications

Multi-Agent MDP Homomorphic Networks

Multi-Agent MDP Homomorphic Networks

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Social physics

Contact Info

Product

Resources

About