DART: Noise Injection for Robust Imitation Learning

Laskey, Michael; Fox, Roy; Dragan, Anca D.; Goldberg, Ken

doi:10.48550/arxiv.1703.09327

Cited by 19 publications

(24 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For Dial, we find that behaviour cloning alone is not able to solve the task and all of our models never generate the maximum return due to the exposure bias. This is consistent with the previous literature on applying behavior cloning to robotics tasks with continuous states (Laskey et al, 2017). More details can be found in Figure 13.…”

Section: Behavior Cloningsupporting

confidence: 92%

Learning Task Decomposition with Ordered Memory Policy Network

Lu,

Shen,

Zhou

et al. 2021

Preprint

View full text Add to dashboard Cite

Many complex real-world tasks are composed of several levels of sub-tasks. Humans leverage these hierarchical structures to accelerate the learning process and achieve better generalization. In this work, we study the inductive bias and propose Ordered Memory Policy Network (OMPN) to discover subtask hierarchy by learning from demonstration. The discovered subtask hierarchy could be used to perform task decomposition, recovering the subtask boundaries in an unstructured demonstration. Experiments on Craft and Dial demonstrate that our model can achieve higher task decomposition performance under both unsupervised and weakly supervised settings, comparing with strong baselines. OMPN can also be directly applied to partially observable environments and still achieve higher task decomposition performance. Our visualization further confirms that the subtask hierarchy can emerge in our model 1 .

show abstract

Section: Behavior Cloningsupporting

confidence: 92%

Learning Task Decomposition with Ordered Memory Policy Network

Lu,

Shen,

Zhou

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Scientifically, this result is valuable evidence about the limitations of pure imitation in the driving domain, especially in light of recent promising results for high-capacity models (Laskey et al (2017a)). But practically, we needed ways to address this challenge without exposing demonstrators to new states actively (Ross et al (2011); Laskey et al (2017b)) or performing reinforcement learning (Kuefler et al (2017)).…”

Section: Introductionmentioning

confidence: 99%

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

Bansal¹,

Krizhevsky²,

Ogale³

2018

Preprint

141

274

View full text Add to dashboard Cite

Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert's driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress -the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the ChauffeurNet model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world.

show abstract

“…Prior work has attempted to reduce the number of human annotations needed [10,23,30] but relabeling is still required. Noise injection during expert demonstrations has also been proposed in order to correct for covariate shift [22]. Other paradigms for human-in-the-loop policy learning include collaboration [14,15], teaching the robot through informative sample selection [7,9,17], and leveraging physical kinesthetic corrections [5,6].…”

Section: Related Workmentioning

confidence: 99%

Human-in-the-Loop Imitation Learning using Remote Teleoperation

Mandlekar¹,

Xu²,

Roberto³

et al. 2020

Preprint

View full text Add to dashboard Cite

Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often contain bottleneck regions that require a sequence of precise actions to make meaningful progress, such as a robot inserting a pod into a coffee machine to make coffee. Trained policies can fail in these regions because small deviations in actions can lead the policy into states not covered by the demonstrations. Intervention-based policy learning is an alternative that can address this issue -it allows human operators to monitor trained policies and take over control when they encounter failures. In this paper, we build a data collection system tailored to 6-DoF manipulation settings, that enables remote human operators to monitor and intervene on trained policies. We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators, and further show that our method outperforms multiple state-ofthe-art baselines for learning from the human interventions on a challenging robot threading task and a coffee making task. Additional results and videos at https://sites.google. com/stanford.edu/iwr

show abstract

DART: Noise Injection for Robust Imitation Learning

Cited by 19 publications

References 12 publications

Learning Task Decomposition with Ordered Memory Policy Network

Learning Task Decomposition with Ordered Memory Policy Network

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

Human-in-the-Loop Imitation Learning using Remote Teleoperation

Contact Info

Product

Resources

About