XIRL: Cross-embodiment Inverse Reinforcement Learning

Zakka, Kevin; Zeng, Andy; Florence, Pete; Tompson, Jonathan; Bohg, Jeannette; Dwibedi, Debidatta

doi:10.48550/arxiv.2106.03911

Cited by 5 publications

(5 citation statements)

References 28 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Peng et al (2020) use an approach based on keypoint matching to learn robotic locomotion behaviors from demonstrations of walking dogs. Zakka et al (2021) learn a visual reward function that allows reinforcement learning agents to learn from demonstrators with different embodiments.…”

Section: Kinematic Retargeting and Visual Teleoperationmentioning

confidence: 99%

Learning dexterity from human hand motion in internet videos

Shaw,

Bahl,

Sivakumar

et al. 2024

The International Journal of Robotics Research

View full text Add to dashboard Cite

To build general robotic agents that can operate in many environments, it is often useful for robots to collect experience in the real world. However, unguided experience collection is often not feasible due to safety, time, and hardware restrictions. We thus propose leveraging the next best thing as real world experience: videos of humans using their hands. To utilize these videos, we develop a method that retargets any 1st person or 3rd person video of human hands and arms into the robot hand and arm trajectories. While retargeting is a difficult problem, our key insight is to rely on only internet human hand video to train it. We use this method to present results in two areas: First, we build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. This enables the robot to collect real-world experience safely using supervision. See these results at https://robotic-telekinesis.github.io . Second, we retarget in-the-wild human internet video into task-conditioned pseudo-robot trajectories to use as artificial robot experience. This learning algorithm leverages action priors from human hand actions, visual features from the images, and physical priors from dynamical systems to pretrain typical human behavior for a particular robot task. We show that by leveraging internet human hand experience, we need fewer robot demonstrations compared to many other methods. See these results at https://video-dex.github.io

show abstract

Section: Kinematic Retargeting and Visual Teleoperationmentioning

confidence: 99%

Learning dexterity from human hand motion in internet videos

Shaw,

Bahl,

Sivakumar

et al. 2024

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…Peng et al [29] use an approach based on keypoint matching to learn robotic locomotion behaviors from demonstrations of walking dogs. Zakka et al [40] learn a visual reward function that allows reinforcement learning agents to learn from demonstrators with different embodiments.…”

Section: Related Workmentioning

confidence: 99%

Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube

Sivakumar¹,

Shaw²,

Pathak³

2022

Preprint

View full text Add to dashboard Cite

We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired humanrobot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots that learn to act autonomously in the real world. The videos can be found at: https://robotic-telekinesis.github.io/

show abstract

“…For example, TCN [39] learns a self-supervised temporal-consistent embedding for imitation learning and reinforcement learning. XIRL [40] learns a self-supervised embedding that estimates task progress for inverse reinforcement learning. [41], [42], [43] directly map the observations such as images to the target domain.…”

Section: A Learning From Noisy Demonstrationmentioning

confidence: 99%

Extraneousness-Aware Imitation Learning

Zheng¹,

Kaizhe²,

Zhecheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Visual imitation learning provides an effective framework to learn skills from demonstrations. However, the quality of the provided demonstrations usually significantly affects the ability of an agent to acquire desired skills. Therefore, the standard visual imitation learning assumes nearoptimal demonstrations, which are expensive or sometimes prohibitive to collect. Previous works propose to learn from noisy demonstrations; however, the noise is usually assumed to follow a context-independent distribution such as a uniform or gaussian distribution. In this paper, we consider another crucial yet underexplored setting -imitation learning with taskirrelevant yet locally consistent segments in the demonstrations (e.g., wiping sweat while cutting potatoes in a cooking tutorial). We argue that such noise is common in real world data and term them as "extraneous" segments. To tackle this problem, we introduce Extraneousness-Aware Imitation Learning (EIL), a self-supervised approach that learns visuomotor policies from third-person demonstrations with extraneous subsequences. EIL learns action-conditioned observation embeddings in a self-supervised manner and retrieves task-relevant observations across visual demonstrations while excluding the extraneous ones. Experimental results show that EIL outperforms strong baselines and achieves comparable policies to those trained with perfect demonstration on both simulated and real-world robot control tasks. The project page can be found here: https://sites.google.com/view/eil-website.

show abstract

XIRL: Cross-embodiment Inverse Reinforcement Learning

Cited by 5 publications

References 28 publications

Learning dexterity from human hand motion in internet videos

Learning dexterity from human hand motion in internet videos

Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube

Extraneousness-Aware Imitation Learning

Contact Info

Product

Resources

About