Variable Impedance Skill Learning for Contact-Rich Manipulation

Abstract

Contact-rich manipulation tasks remain a hard problem in robotics that requires interaction with unstructured environments. Reinforcement Learning (RL) is one potential solution to such problems, as it has been successfully demonstrated on complex continuous control tasks. Nevertheless, current state-ofthe-art methods require policy training in simulation to prevent undesired behavior and later domain transfer even for simple skills involving contact. In this paper, we address the problem of learning contact-rich manipulation policies by extending an existing skill-based RL framework with a variable impedance action space. Our method leverages a small set of suboptimal demonstration trajectories and learns from both position, but also crucially impedance-space information. We evaluate our method on a number of peg-in-hole task variants with a Franka Panda arm and demonstrate that learning variable impedance actions for RL in Cartesian space can be deployed directly on the real robot, without resorting to learning in simulation.

Cauchy DMP: Improving 3C industrial assembly quality with the Cauchy kernel and singular value decomposition

“…The effects of our proposed algorithm are demonstrated through several experiments including improved stability of the algorithm and reduced time consumption of trajectory recurrence in 3C industrial production. There are many other robot precision operations similar to the 3C industry, such as surgical robots [29] and contact‐rich assembly [30]. Our future work will continue to focus on the precise operation of robots in industrial scenarios.…”

SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks

“…However, the direction of motion of the robot will not change unless large contact forces are generated, which means that the actual trajectory executed by the robot may not be optimal with respect to energy and time. Furthermore, some scholars [6], [15], [16] learn different impedance parameters from collected expert demonstration trajectories. However, it is difficult to demonstrate trajectories for tasks like ours, i.e., blind maze exploration.…”

“…Specifically, Martin-Martin et al [7] compared the effect of different action spaces in RL with variable impedance control in end-effector space for contactrich tasks, such as surface-wiping and door-opening. Some work [4], [5], [6] leveraged demonstrations to learn variable impedance parameters as part of the action space. On the other hand, a safe action or policy can be learned with RL to significantly enhance safety during the training stage.…”

“…Learning-based approaches became a powerful way to address this challenge, with a wide variety of methods for tuning impedance parameters [2]. In previous works, such parameters were learned from human (or expert) demonstrations using Dynamic movement primitives (DMPs) [4], inverse reinforcement learning (IRL) [5] and skill priors [6]; however, providing demonstrations in advance is relatively time-consuming and labor-intensive. Also, it can be impractical to provide expert demonstrations in some tasks where the environment is unstructured or potentially hostile.…”

Learn from Robot: Transferring Skills for Diverse Manipulation via Cycle Generative Networks

“…This model consists of a skill encoder q(z|a), which generates the latent skill actions z ∼ N (µ z , σ z ), and a decoder p dec (a|z), responsible for predicting a sequence of low-level actions a = {a t , • • • , a t+H−1 }, where H ∈ N + denotes the action horizon. Similar to the approach in [28], [29], we train the VAE by optimizing the evidence lower bound (ELBO) given by the equation:…”