Experience Reuse with Probabilistic Movement Primitives

Abstract

Acquiring new robot motor skills is cumbersome, as learning a skill from scratch and without prior knowledge requires the exploration of a large space of motor configurations. Accordingly, for learning a new task, time could be saved by restricting the parameter search space by initializing it with the solution of a similar task. We present a framework which is able of such knowledge transfer from already learned movement skills to a new learning task. The framework combines probabilistic movement primitives with descriptions of their effects for skill representation. New skills are first initialized with parameters inferred from related movement primitives and thereafter adapted to the new task through relative entropy policy search. We compare two different transfer approaches to initialize the search space distribution with data of known skills with a similar effect. We show the different benefits of the two knowledge transfer approaches on an object pushing task for a simulated 3-DOF robot. We can show that the quality of the learned skills improves and the required iterations to learn a new task can be reduced by more than 60% when past experiences are utilized.

Learning Task-Parameterized Skills From Few Demonstrations

“…As a consequence, the resulting TP-GMM will not able to resolve on the corner cases. RL-based policy search maybe promising in resolving corner cases as reported in ACNMP framework in [31] and adaptive ProMP in [32]. However, for dressing tasks, the reward needs to be carefully designed as reflected in [33].…”

Learning Task-Parameterized Skills from Few Demonstrations

“…As a consequence, the resulting TP-GMM will not able to resolve on the corner cases. RLbased policy search maybe promising in resolving corner cases as reported in ACNMP framework in [31] and adaptive ProMP in [32]. However, for dressing tasks, the reward needs to be carefully designed as reflected in [33].…”