Experience Reuse with Probabilistic Movement Primitives

Experience Selection Using Dynamics Similarity for Efficient Multi-Source Transfer Learning Between Robots

“…Knowledge transfer or transfer learning has been used in robotics to accelerate the learning process for acquiring new skills, or to further improve a robots' performance (e.g., [1], [2]). At its core, transfer learning seeks to leverage existing experiences to initialize learning in a new domain that would otherwise be learned from scratch, thereby reducing the time and cost for training on physical or virtual robots.…”

“…At its core, transfer learning seeks to leverage existing experiences to initialize learning in a new domain that would otherwise be learned from scratch, thereby reducing the time and cost for training on physical or virtual robots. Examples of transfer learning in robotics include both (i) inter-task transfer learning, in which the experience learned for one task is used for learning new tasks on the same robot [2]- [4], and (ii) inter-robot transfer learning, where the experience from one robot is used to accelerate or improve the learning on a second robot [5]- [7].…”

“…II. RELATED WORK Transfer learning has been considered in a wide range of robot learning problems, which include, but are not limited to, robot model learning [5], [14], [15], policy learning [1], [2], [16], and objective function learning [17], [18]. Whether it is inter-task transfer [2], [17], inter-robot transfer [5], [14]- [16], or transferring experience from simulation to reality [18], [19], the common goal in transfer learning is to leverage existing experiences to accelerate or improve the learning process of the target task or robot.…”

“…While in an inter-task transfer framework there exist discussions on characterizing task similarity for multi-source experience selection via a maximum likelihood model selection approach [17], or based on the difference of task descriptor representations [2], discussions on a similarity characterization of robot dynamics for inter-robot transfer problems are rare. As similarity and transfer performance are positively correlated concepts [8], [10], in inter-robot transfer problems, especially those involving robot model learning, characterizing the dynamics similarity of the robots is essential for predicting the effectiveness of transferring an experience.…”

“…As similarity and transfer performance are positively correlated concepts [8], [10], in inter-robot transfer problems, especially those involving robot model learning, characterizing the dynamics similarity of the robots is essential for predicting the effectiveness of transferring an experience. In this work, we consider a multi-source transfer learning setup similar to [2], [17], but we focus on an interrobot model transfer problem, where our goal is to select the best source robot to facilitate the learning of a target robot.…”