iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes

Shen, Bokui; Xia, Fei; Li, Chengshu; Martí­n-Martí­n, Roberto; Fan, Linxi; Wang, Guanzhi; Pérez-D’Arpino, Claudia; Buch, Shyamal; Srivastava, Sanvesh; Tchapmi, Lyne P.; Tchapmi, Micael; Vainio, Kent; Wong, Josiah; Li, Feifei; Savarese, Silvio

doi:10.1109/iros51168.2021.9636667

Cited by 107 publications

(78 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Habitat [3] Magnum 3D scans none continuous navigation (navmesh) building-scale 3,000 AI2-THOR [6] Unity Unity Unity rigid dynamics, animated interactions room-scale 30 -60 ManipulaTHOR [26] Unity Unity Unity AI2-THOR + manipulation room-scale 30 -40 ThreeDWorld [7] Unity Unity Unity (PhysX) + FLEX rigid + particle dynamics room/house-scale 5 -168 SAPIEN [34] OpenGL/OptiX configurable PhysX rigid/articulated dynamics object-level 200 -400 † RLBench [35] CoppeliaSim (OpenGL) Gouraud shading CoppeliaSim (Bullet/ODE) rigid/articulated dynamics table-top 1 -60 † iGibson [36] PyRender PBR shading PyBullet rigid/articulated dynamics house-scale 100…”

Section: Physicsmentioning

confidence: 99%

“…How are interactive simulators built today? Either by relying on game engines [6,50,51] or via a 'homebrew' integration of existing rendering and physics libraries [7,34,36,52]. Both options have problems.…”

Section: Related Workmentioning

confidence: 99%

“…Our 'Idle' setting is similar to the benchmarking setup of iGibson [36], which reports 100 SPS. In contrast, H2.0 single-process with all optimizations turned off is 240% faster (242 vs 100 SPS).…”

Section: Benchmarkingmentioning

confidence: 99%

See 2 more Smart Citations

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Szot¹,

Clegg²,

Undersander³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack -data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850× real-time) on an 8-GPU node, representing 100× speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, prepare groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from 'hand-off problems', and (3) SPA pipelines are more brittle than RL policies. Figure 1: A mobile manipulator (Fetch robot) simulated in Habitat 2.0 performing rearrangement tasks in a ReplicaCAD apartment -(left) opening a drawer before picking up an item from it, and (right) placing an object into the bowl after navigating to the table. Best viewed in motion at https://sites.google.com/view/habitat2. Preprint. Under review.

show abstract

Section: Physicsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Szot¹,

Clegg²,

Undersander³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this way, action is viewed as a tuple A = action, object, constraints , and a task is thus an action sequence T = {A 1 , ..., A n }. As learning from real-world domestic environments is costprohibitive, many researchers proposed simulation environments for household tasks [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]. Despite that all of them provide "interactive" functionalities, the supports for the agent actions are limited in three aspects, namely action types, object types, and interaction physics.…”

Section: Introductionmentioning

confidence: 99%

“…From the action side, previous environments generally built around simple atomic actions like navigation [2], [6], grasping, transporting, and placing (or dropping) [1], [7], [8], [10]. However, for some environments [1], [14], even these simple actions are usually implemented in an abstract, not physics-based way.…”

Section: Introductionmentioning

confidence: 99%

RFUniverse: A Physics-based Action-centric Interactive Environment for Everyday Household Tasks

Haoyuan¹,

Wang²,

Han³

et al. 2022

Preprint

View full text Add to dashboard Cite

Household environments are important testbeds for embodied AI research. Many simulation environments have been proposed to develop learning models for solving everyday household tasks. However, though interactions are paid attention to in most environments, the actions operating on the objects are not well supported concerning action types, object types, and interaction physics. To bridge the gap at the action level, we propose a novel physics-based actioncentric environment, RFUniverse, for robot learning of everyday household tasks. RFUniverse supports interactions among 87 atomic actions and 8 basic object types in a visually and physically plausible way. To demonstrate the usability of the simulation environment, we perform learning algorithms on various types of tasks, namely fruit-picking, cloth-folding and sponge-wiping for manipulation, stair-chasing for locomotion, room-cleaning for multi-agent collaboration, milk-pouring for task and motion planning, and bimanual-lifting for behavior cloning from VR interface. Client-side Python APIs, learning codes, models, and the database will be released. Demo video for atomic actions can be found in supplementary materials: https://sites.google.com/view/rfuniverse

show abstract

An Open-Source Multi-goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

Yang

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes

Cited by 107 publications

References 38 publications

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

RFUniverse: A Physics-based Action-centric Interactive Environment for Everyday Household Tasks

An Open-Source Multi-goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

Contact Info

Product

Resources

About