Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control

Hermann, Lukás; Argus, Max; Eitel, Andreas; Amiranashvili, Artemij; Burgard, Wolfram; Brox, Thomas

doi:10.1109/icra40945.2020.9197108

Cited by 22 publications

(19 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stacking is widely used as a benchmark task for robotics RL since it comprises important subskills like reaching, grasping and placing. We build upon a PyBullet [41] simulation environment for block stacking described in [4], where a 7-DOF Kuka iiwa robot with a parallel gripper has to pick up a small cube and place it on top of another cube.…”

Section: Methodsmentioning

confidence: 99%

“…The environment yields a sparse reward upon task completion. Since exploration for long horizon sparse reward tasks is challenging for RL algorithms, we follow [4] and use Adaptive Curriculum Generation from Demonstrations (ACGD) to guide exploration with a handful of demonstrations.…”

Section: Methodsmentioning

confidence: 99%

“…This helps to learn a policy that is invariant to these shifts and is more likely to succeed on the transfer from simulated to real images or with varying backgrounds in robotic environments. It has led to many successful zero-shot sim2real transfer applications such as indoor flight [1], grasping [2], [3], stacking [4], [5] or dexterous in-hand manipulation [6], [7]. However, adding domain randomization hampers the DRL training process in simulation.…”

Section: Introductionmentioning

confidence: 99%

“…The task difficulty is increased by training and evaluating the stacking on randomly colored backgrounds. We apply the DRAP initialization to the ACGD algorithm [4] and show that DRAP improves the stacking success rate in simulation and enables the transfer to the real robot with differently colored and textured backgrounds.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

Amiranashvili,

Argus,

Hermann

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Visual domain randomization in simulated environments is a widely used method to transfer policies trained in simulation to real robots. However, domain randomization and augmentation hamper the training of a policy. As reinforcement learning struggles with a noisy training signal, this additional nuisance can drastically impede training. For difficult tasks it can even result in complete failure to learn. To overcome this problem we propose to pre-train a perception encoder that already provides an embedding invariant to the randomization. We demonstrate that this yields consistently improved results on a randomized version of DeepMind control suite tasks and a stacking environment on arbitrary backgrounds with zero-shot transfer to a physical robot.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

Amiranashvili,

Argus,

Hermann

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These data-driven approaches are trained by optimizing directly for skill success. In particular, some recent works have proposed learning robot skills such as grasping [24], pick-and-stow [25], and part discovery [26] first in simulation, where interaction is cheap and labeled, and then transferring the agent to the real world. Another transfer learning framework adoption is to learn a vision model from passive observations first and then to leverage the learned representations for learning manipulation skill models [27].…”

Section: Related Workmentioning

confidence: 99%

Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

Nematollahi¹,

Rosete-Beas²,

Röfer³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through trajectories rather than making instantaneous decisions individually at each time step. To this end, we propose the Soft Actor-Critic Gaussian Mixture Model (SAC-GMM), a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribution space through interactions with the environment. Our approach combines classical robotics techniques of learning from demonstration with the deep reinforcement learning framework and exploits their complementary nature. We show that our method utilizes sensors solely available during the execution of preliminarily learned skills to extract relevant features that lead to faster skill refinement. Extensive evaluations in both simulation and real-world environments demonstrate the effectiveness of our method in refining robot skills by leveraging physical interactions, high-dimensional sensory data, and sparse task completion rewards. Videos, code, and pre-trained models are available at http://sac-gmm.cs.uni-freiburg.de.

show abstract

Learning Pre-grasp Pushing Manipulation of Wide and Flat Objects Using Binary Masks

Zhong

2021

Neural Information Processing

View full text Add to dashboard Cite

Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control

Cited by 22 publications

References 16 publications

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

Learning Pre-grasp Pushing Manipulation of Wide and Flat Objects Using Binary Masks

Contact Info

Product

Resources

About