Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets

Ebert, Frederik; Yang, Yanlai; Schmeckpeper, Karl; Bucher, Bernadette; Georgakis, Georgios V.; Daniilidis, Kostas; Finn, Chelsea; Levine, Sergey

doi:10.48550/arxiv.2109.13396

Cited by 11 publications

(32 citation statements)

References 22 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent research suggests that learning from diverse and large-scale dataset is a promising path towards generalizable manipulation [9,14,27]. Examples of such dataset include demonstrations across domains [9,14,20,24], mixture of exploration and expert data [2], and instruction-conditional data [27].…”

Section: Representation Learning For Generalizable Manipulationmentioning

confidence: 99%

Generalizable Task Planning through Representation Pretraining

Wang¹,

Xu²,

Li³

2022

Preprint

View full text Add to dashboard Cite

The ability to plan for multi-step manipulation tasks in unseen situations is crucial for future home robots. But collecting sufficient experience data for end-to-end learning is often infeasible in the real world, as deploying robots in many environments can be prohibitively expensive. On the other hand, large-scale scene understanding datasets contain diverse and rich semantic and geometric information. But how to leverage such information for manipulation remains an open problem. In this paper, we propose a learning-to-plan method that can generalize to new object instances by leveraging object-level representations extracted from a synthetic scene understanding dataset. We evaluate our method with a suite of challenging multi-step manipulation tasks inspired by household activities [28] and show that our model achieves measurably better success rate than state-of-the-art end-to-end approaches. Additional information can be found at: https: //sites.google.com/view/gentp

show abstract

Section: Representation Learning For Generalizable Manipulationmentioning

confidence: 99%

Generalizable Task Planning through Representation Pretraining

Wang¹,

Xu²,

Li³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A supplementary video visualizing examples of distances predicted by HOLD-C as well as policy roll-outs is included on the project website 2 . We also qualitatively evaluate the predictions on episodes from Bridge Data [40], a diverse dataset of robot manipulation tasks recorded on real robots, and include examples in the video. The results suggest our distance model may well generalize to training manipulation policies on real robots.…”

Section: Overview Of Supplementary Materialsmentioning

confidence: 99%

Learning Reward Functions for Robotic Manipulation by Observing Humans

Alakuijala¹,

Dulac-Arnold²,

Mairal³

et al. 2022

Preprint

View full text Add to dashboard Cite

Observing a human demonstrator manipulate objects provides a rich, scalable and inexpensive source of data for learning robotic policies. However, transferring skills from human videos to a robotic manipulator poses several challenges, not least a difference in action and observation spaces. In this work, we use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. Thanks to the diversity of this training data, the learned reward function sufficiently generalizes to image observations from a previously unseen robot embodiment and environment to provide a meaningful prior for directed exploration in reinforcement learning. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective. By conditioning the function on a goal image, we are able to reuse one model across a variety of tasks. Unlike prior work on leveraging human videos to teach robots, our method, Human Offline Learned Distances (HOLD) requires neither a priori data from the robot environment, nor a set of taskspecific human demonstrations, nor a predefined notion of correspondence across morphologies, yet it is able to accelerate training of several manipulation tasks on a simulated robot arm compared to using only a sparse reward obtained from task completion.

show abstract

“…Inspired by progress catalyzed by datasets like Ima-geNet [23], there have been efforts to collect large datasets for robot manipulation [6,9,13,21,29]. However, most do not fully address the problem of different robot hardware, and often collect data with only one type of robot model.…”

Section: Related Workmentioning

confidence: 99%

Tool as Embodiment for Recursive Manipulation

Noguchi¹,

Matsushima²,

Matsuo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans and many animals exhibit a robust capability to manipulate diverse objects, often directly with their bodies and sometimes indirectly with tools. Such flexibility is likely enabled by the fundamental consistency in underlying physics of object manipulation such as contacts and force closures. Inspired by viewing tools as extensions of our bodies, we present Tool-As-Embodiment (TAE), a parameterization for tool-based manipulation policies that treat hand-object and tool-object interactions in the same representation space. The result is a single policy that can be applied recursively on robots to use end effectors to manipulate objects, and use objects as tools, i.e. new endeffectors, to manipulate other objects. By sharing experiences across different embodiments for grasping or pushing, our policy exhibits higher performance than if separate policies were trained. Our framework could utilize all experiences from different resolutions of tool-enabled embodiments to a single generic policy for each manipulation skill. Videos at https://sites.google.com/ view/recursivemanipulation

show abstract

Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets

Cited by 11 publications

References 22 publications

Generalizable Task Planning through Representation Pretraining

Generalizable Task Planning through Representation Pretraining

Learning Reward Functions for Robotic Manipulation by Observing Humans

Tool as Embodiment for Recursive Manipulation

Contact Info

Product

Resources

About