Task-level imitation learning using variance-based movement optimization

Abstract

Recent advances in the field of humanoid robotics increase the complexity of the tasks that such robots can perform. This makes it increasingly difficult and inconvenient to program these tasks manually. Furthermore, humanoid robots, in contrast to industrial robots, should in the distant future behave within a social environment. Therefore, it must be possible to extend the robot's abilities in an easy and natural way. To address these requirements, this work investigates the topic of imitation learning of motor skills. The focus lies on providing a humanoid robot with the ability to learn new bimanual tasks through the observation of object trajectories. For this, an imitation learning framework is presented, which allows the robot to learn the important elements of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models. The learned information is used to initialize an attractor-based movement generation algorithm that optimizes the reproduced movement towards the fulfillment of additional criteria, such as collision avoidance. Experiments performed with the humanoid robot ASIMO show that the proposed system is suitable for transferring information from a human demonstrator to the robot. These results provide a good starting point for more complex and interactive learning tasks.beverage from a bottle into a glass by observing a teacher demonstrating this task. This choice is arbitrary and the researched methods do not depend on this specific choice but are general. An overview of the whole imitation learning process is depicted in figure 1 and explained in detail within the upcoming sections.

Learning and Reproduction of Gestures by Imitation

“…We also showed that it allows simultaneous consideration of constraints in joint space and task space [23]. Muehlig et al [24] recently extended the GMR approach to learn bimanual skills by imitation. In this work, the authors used GMR as a compact probabilistic representation of the task constraints which is then used during reproduction by a gradient-based trajectory optimizer.…”

Motion learning in variable environments using probabilistic flow tubes

“…The demonstrated sequences may have different numbers of data entries, so we use dynamic time warping again to temporally match each of the normalized demonstrated sequences in S to the mean sequence s ef f , and interpolate so that all have the same number of data entries (lines [8][9][10][11]. Finally, we compute covariances at each corresponding time step across the temporally matched normalized demonstrated sequences (lines 12-13).…”

“…We also compared our approach to a recent approach by Mühlig et al [9]. Mühlig's approach also uses dynamic time warping to temporally match demonstrated trajectories, but uses Gaussian mixture models (GMM) to describe learned motions.…”

“…Researchers at USC [7], [8] modeled learned motions as a spring system described by differential equations with parameterized start and goal locations. In contrast, Mühlig et al [9] chose to abstract the demonstrated time-series data in task space with Gaussian Mixture Models. Calinon et al [10] model motions as a mixture of Gaussian Mixture Models and Bernoulli distributions that captures both spatial and temporal aspects of a motion.…”

Robot feedback shapes the tutor’s presentation

“…The experimental platform used was the Honda Humanoid Research Robot, a 1.20 m sized humanoid robot set up to run autonomously (Mühlig 2009). To enable the robot to detect and follow the tutor's hand movements and the object's position and trajectories, marker-based tracking methods were used.…”