Visual Room Rearrangement

Weihs, Luca; Matt, Deitke,; Kembhavi, Aniruddha; Mottaghi, Roozbeh

doi:10.48550/arxiv.2103.16544

Cited by 2 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(pick-and-place), and joint configuration of articulated objects. VisualRoom Rearrangement [24] includes objects that can be broken (changing object geometry).…”

Section: A31 Selection Of 100 Activities For Behaviormentioning

confidence: 99%

“…Recently, significant progress has been made in developing solutions to embodied AI problems such as (visual) navigation [1][2][3][4][5], interactive Q&A [6][7][8][9][10], instruction following [11][12][13][14][15], and manipulation [16][17][18][19][20][21][22]. To calibrate the progress, several lines of pioneering efforts have been made towards benchmarking embodied AI in simulated environments, including Rearrangement [23,24], TDW Transport Challenge [25], VirtualHome [26], ALFRED [11], Interactive Gibson Benchmark [27], MetaWorld [28], and RLBench [29], among others [30][31][32]). These efforts are inspiring, but their activities represent only a fraction of challenges that humans face in their daily lives.…”

Section: Introductionmentioning

confidence: 99%

“…Closer to BEHAVIOR, a recent group of benchmarks has focused on rearrangement tasks [23][24][25] in realistic simulation environments with diverse scenes. The initial Rearrangement position paper [23] poses critical questions such as how to define embodied AI tasks and measure solution quality.…”

Section: Introductionmentioning

confidence: 99%

“…Figure A.2: Statistics of the 100 activities in BEHAVIOR: a) Distribution of simulatable activities in the American Time Use Survey (left axis) and BEHAVIOR (right axis) based on categories from the survey -BEHAVIOR covers a realistic distribution of activities; b) Cumulative visualization of activities enabled by different types of state changes in BEHAVIOR with comparison to recent prior work -based on requirements, some activities could be considered transport/rearrangement (blue) or visual-room rearrangement (blue and green), while others are out of their scope (red); c) We visualize the specific requirements for each of the BEHAVIOR activities, with the same coloring scheme as in b). Activities in BEHAVIOR present significantly more diverse requirements than prior work focused on transport/rearrangement tasks[23,25,24] enabling the evaluation of more general embodied AI solutions.…”

mentioning

confidence: 99%

See 3 more Smart Citations

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

Srivastava¹,

Li²,

Lingelbach³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation, spanning a range of everyday household chores such as cleaning, maintenance, and food preparation. These activities are designed to be realistic, diverse and complex, aiming to reproduce the challenges that agents must face in the real world. Building such a benchmark poses three fundamental difficulties for each activity: definition (it can differ by time, place, or person), instantiation in a simulator, and evaluation. BEHAVIOR addresses these with three innovations. First, we propose an object-centric, predicate logic-based description language for expressing an activity's initial and goal conditions, enabling generation of diverse instances for any activity. Second, we identify the simulator-agnostic features required by an underlying environment to support BEHAVIOR, and demonstrate its realization in one such simulator. Third, we introduce a set of metrics to measure task progress and efficiency, absolute and relative to human demonstrators. We include 500 human demonstrations in virtual reality (VR) to serve as the human ground truth. Our experiments demonstrate that even state-of-the-art embodied AI solutions struggle with the level of realism, diversity, and complexity imposed by the activities in our benchmark. We make BEHAVIOR publicly available at behavior.stanford.edu to facilitate and calibrate the development of new embodied AI solutions.

show abstract

“…(pick-and-place), and joint configuration of articulated objects. VisualRoom Rearrangement [24] includes objects that can be broken (changing object geometry).…”

Section: A31 Selection Of 100 Activities For Behaviormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

Srivastava¹,

Li²,

Lingelbach³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The main function of these simulators is to compute the motion resulting from the physical contact-interaction between (rigid) bodies, as this is the main process that allows robots to navigate and manipulate the environment. This kinodynamic simulation is sufficient for pick-and-place and rearrangement tasks [11,12,13,14]; however, as the field advances, researchers are taking on more diverse and complex tasks that cannot be performed in these simulators, e.g., household activities that involve changing the temperature of objects, their dirtiness and wetness levels. There is a need for new simulation environments that can maintain and update new types of object states to broaden the diversity of activities that can be studied.…”

Section: Introductionmentioning

confidence: 99%

iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks

Li¹,

Xia²,

Martín-Martín³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent research in embodied AI has been boosted by the use of simulation environments to develop and train robot learning approaches. However, the use of simulation has skewed the attention to tasks that only require what robotics simulators can simulate: motion and physical contact. We present iGibson 2.0, an open-source simulation environment that supports the simulation of a more diverse set of household tasks through three key innovations. First, iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Additionally, given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. This functionality can generate potentially infinite instances of tasks with minimal effort from the users. The sampling mechanism allows our scenes to be more densely populated with small objects in semantically meaningful locations. Third, iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations. As a result, we can collect demonstrations from humans on these new types of tasks, and use them for imitation learning. We evaluate the new capabilities of iGibson 2.0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new directions of research in embodied AI. iGibson 2.0 and its new dataset are publicly available at http://svl.stanford.edu/igibson/.

show abstract

Visual Room Rearrangement

Cited by 2 publications

References 35 publications

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks

Contact Info

Product

Resources

About