Residual Learning From Demonstration: Adapting DMPs for Contact-Rich Manipulation

Abstract

Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC) but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contactrich manipulation. However, it is not clear how to best do this with DMPs. As a result, we consider several possible ways for adapting a DMP formulation and propose "residual Learning from Demonstration" (rLfD), a framework that combines DMPs with Reinforcement Learning (RL) to learn a residual correction policy. Our evaluations suggest that applying residual learning directly in task space and operating on the full pose of the robot can significantly improve the overall performance of DMPs. We show that rLfD offers a gentle to the joints solution that improves the task success and generalisation of DMPs and enables transfer to different geometries and frictions through few-shot task adaptation. The proposed framework is evaluated on a set of tasks. A simulated robot and a physical robot have to successfully insert pegs, gears and plugs into their respective sockets. Other material and videos accompanying this paper are provided at https://sites.google.com/view/rlfd/.

Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments

“…Recent success in deep RL [7], [10], [13], [14] enables a robot to learn complex manipulation tasks such as grasping [7], [8] and insertion [9], [15], [16], [17], [18] driven by a reward. To avoid the requirement of specialist knowledge for reward engineering, several prior works have proposed sample-efficient RL methods that can learn complex manipulation skills from a sparse reward by leveraging a small number of demonstrations for guided exploration [8], [9], [19], [15].…”

AL-ProMP: Force-relevant skills learning and generalization method for robotic polishing

“…Contact tasks skills learning from demonstration In robotic contact tasks, the robot's position [20], orientation [21], contact force [22][23] and impedance [24][25] [26] information are closely related to the skills involved. Typically, DMPs [27][28] are used to learn skills related to robot position and orientation data, with orientation data commonly represented in the form of quaternions.…”

InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory Input

“…Residual policy methods [9], [2] use a pre-program fixed policy while employing RL only to learn a correction policy. LfD is used to derive a baseline policy [10], [1] or to initiate the learning process. While the above-noted approaches have shown significant results in very challenging problems, they are limited by the online RL scheme's ability to collect a large variety of data on the robot and utilize a large collected experience.…”