Model-Based Inverse Reinforcement Learning from Visual Demonstrations

Das, Neha; Bechtle, Sarah; Davchev, Todor; Jayaraman, Dinesh; Rai, Akshara; Meier, Franziska

doi:10.48550/arxiv.2010.09034

Cited by 7 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Agents can utilize observations of exemplar behavior in order to learn aspects of the desired model. For instance, agents can learn actions or skills from observed imagery via the inclusion of a learned cost function [24,124] or by clustering observations into skills [22]. Other recent results have focused on analytical/theoretical aspects of the problem relating to reward function search or aspects of the behavior policy providing the demonstrations [10,50,79,100].…”

Section: Applications and Recent Resultsmentioning

confidence: 99%

Modeling Human Behavior Part I -- Learning and Belief Approaches

Fuchs¹,

Passarella²,

Conti³

2022

Preprint

View full text Add to dashboard Cite

There is a clear desire to model and comprehend human behavior. Trends in research covering this topic show a clear assumption that many view human reasoning as the presupposed standard in artificial reasoning. As such, topics such as game theory, theory of mind, machine learning, etc. all integrate concepts which are assumed components of human reasoning. These serve as techniques to attempt to both replicate and understand the behaviors of humans. In addition, next generation autonomous and adaptive systems will largely include AI agents and humans working together as teams. To make this possible, autonomous agents will require the ability to embed practical models of human behavior, which allow them not only to replicate human models as a technique to "learn", but to to understand the actions of users and anticipate their behavior, so as to truly operate in symbiosis with them. The main objective of this paper it to provide a succinct yet systematic review of the most important approaches in two areas dealing with quantitative models of human behaviors. Specifically, we focus on (i) techniques which learn a model or policy of behavior through exploration and feedback, such as Reinforcement Learning, and (ii) directly model mechanisms of human reasoning, such as beliefs and bias, without going necessarily learning via trial-and-error.CCS Concepts: • Human-centered computing; • General and reference → Surveys and overviews;

show abstract

Section: Applications and Recent Resultsmentioning

confidence: 99%

Modeling Human Behavior Part I -- Learning and Belief Approaches

Fuchs¹,

Passarella²,

Conti³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…a) Intermediate representations: Several prior works have addressed learning robotic policies from human videos via intermediate representations such as pose estimation or keypoint tracking [20]- [22]. In this work, our aim is to advance the capabilities of learning from raw video data, without depending on hand-crafted intermediate representations of human hands or an object database.…”

Section: Related Workmentioning

confidence: 99%

Learning Reward Functions for Robotic Manipulation by Observing Humans

Alakuijala¹,

Dulac-Arnold²,

Mairal³

et al. 2022

Preprint

View full text Add to dashboard Cite

Observing a human demonstrator manipulate objects provides a rich, scalable and inexpensive source of data for learning robotic policies. However, transferring skills from human videos to a robotic manipulator poses several challenges, not least a difference in action and observation spaces. In this work, we use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. Thanks to the diversity of this training data, the learned reward function sufficiently generalizes to image observations from a previously unseen robot embodiment and environment to provide a meaningful prior for directed exploration in reinforcement learning. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective. By conditioning the function on a goal image, we are able to reuse one model across a variety of tasks. Unlike prior work on leveraging human videos to teach robots, our method, Human Offline Learned Distances (HOLD) requires neither a priori data from the robot environment, nor a set of taskspecific human demonstrations, nor a predefined notion of correspondence across morphologies, yet it is able to accelerate training of several manipulation tasks on a simulated robot arm compared to using only a sparse reward obtained from task completion.

show abstract

“…Prior work in visual manipulation has involved learning predictive models from videos of a robot interacting with its environment. These models are used to find actions with simulation roll-outs, in a model-predictive fashion [16,17,1,18,19,20,21]. These approaches have been successful at solving prehensile manipulation problems where the dynamics are hard to model.…”

Section: Related Workmentioning

confidence: 99%

A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation

Aceituno¹,

Rodríguez²,

Tulsiani³

et al. 2021

Preprint

View full text Add to dashboard Cite

Specifying tasks with videos is a powerful technique towards acquiring novel and general robot skills. However, reasoning over mechanics and dexterous interactions can make it challenging to scale learning contact-rich manipulation. In this work, we focus on the problem of visual non-prehensile planar manipulation: given a video of an object in planar motion, find contact-aware robot actions that reproduce the same object motion. We propose a novel architecture, Differentiable Learning for Manipulation (DLM), that combines video decoding neural models with priors from contact mechanics by leveraging differentiable optimization and finite difference based simulation. Through extensive simulated experiments, we investigate the interplay between traditional model-based techniques and modern deep learning approaches. We find that our modular and fully differentiable architecture performs better than learning-only methods on unseen objects and motions. https://github.com/baceituno/dlm.

show abstract

Model-Based Inverse Reinforcement Learning from Visual Demonstrations

Cited by 7 publications

References 15 publications

Modeling Human Behavior Part I -- Learning and Belief Approaches

Modeling Human Behavior Part I -- Learning and Belief Approaches

Learning Reward Functions for Robotic Manipulation by Observing Humans

A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation

Contact Info

Product

Resources

About