Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments

Abstract

We present Interactive Gibson, the first comprehensive benchmark for training and evaluating Interactive Navigation: robot navigation strategies where physical interaction with objects is allowed and even encouraged to accomplish a task. For example, the robot can move objects if needed in order to clear a path leading to the goal location. Our benchmark comprises two novel elements: 1) a new experimental setup, the Interactive Gibson Environment, which simulates high fidelity visuals of indoor scenes, and high fidelity physical dynamics of the robot and common objects found in these scenes; 2) a set of Interactive Navigation metrics which allows one to study the interplay between navigation and physical interaction. We present and evaluate multiple learning-based baselines in Interactive Gibson, and provide insights into regimes of navigation with different trade-offs between navigation path efficiency and disturbance of surrounding objects. We make our benchmark publicly available 3 and encourage researchers from all disciplines in robotics (e.g. planning, learning, control) to propose, evaluate, and compare their Interactive Navigation solutions in Interactive Gibson.

Learning Human Search Behavior from Egocentric Visual Inputs

“…To increase complexity of the scene, we add a few open cabinets in which the target objects can be placed. We use iGibson [XSL*20] environment that provides 3D scans reconstructed from realistic indoor environments and a photorealistic renderer to generate vision inputs to the character. For physics simulation, we use PyBullet [CB17] to simulate the motion of abstract model and to check collision with the environment.…”

“…Our method also utilizes the simulation tools developed by other researchers to train our virtual human in a realistic environment. Specifically, we used iGibson, which provides a suite of realistic indoor environments [XSL*20]. Unlike prior work in visual navigation that focus on agents moving in 2D or 2.5D spaces, e.g.…”

“…However, directly training visual navigation agents in real environments is expensive, especially since we usually want to train agents for a large variety of environments. To this end, researchers have developed simulated environments that leverages modern 3D scanning techniques to reproduce real‐world scenarios and allow agents to observe photo‐realistic visual inputs in a scalable way [XSL*20, MAO*19, KMH*17]. These tools enables rapid advancements in learning algorithms and neural network structures in training visual navigation agents [GDL*17, PS17, ZTB*17, FTFFS19, WKM*20].…”

Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation

“…4), we create five realistic multi-stage simulated household kitchen tasks and collect a large-scale multi-user demonstration dataset. We design the simulated tasks in a realistic kitchen environment using PyBullet [60] and the iGibson [61,62] framework with a Fetch [63] robot that must manipulate a bowl. Across all tasks, the robot's initial pose and bowl location is randomized between episodes.…”

A benchmark with decomposed distribution shifts for 360 monocular depth estimation

“…Our benchmark relies on two recent 3D building scan datasets, Matterport3D (M3D) [10] and GibsonV2 (GV2) [11], using modern synthesis to produce high quality spherical panoramas coupled with depth maps. Sample images can be found in Figure 2.…”