From Motor Learning to Interaction Learning in Robots

Abstract

-The number of advanced robot systems has been increasing in recent years yielding a large variety of versatile designs with many degrees of freedom. These robots have the potential of being applicable in uncertain tasks outside well-structured industrial settings. However, the complexity of both systems and tasks is often beyond the reach of classical robot programming methods. As a result, a more autonomous solution for robot task acquisition is needed where robots adaptively adjust their behaviour to the encountered situations and required tasks. Learning approaches pose one of the most appealing ways to achieve this goal. However, while learning approaches are of high importance for robotics, we cannot simply use off-the-shelf methods from the machine learning community as these usually do not scale into the domains of robotics due to excessive computational cost as well as a lack of scalability. Instead, domain appropriate approaches are needed. We focus here on several core domains of robot learning. For accurate task execution, we need motor learning capabilities. For fast learning of the motor tasks, imitation learning offers the most promising approach. Self improvement requires reinforcement learning approaches that scale into the domain of complex robots. Finally, for efficient interaction of humans with robot systems, we will need a form of interaction learning. This contribution provides a general introduction to these issues and briefly presents the contributions of the related book chapters to the corresponding research topics.

Autonomous online learning of velocity kinematics on the iCub: A comparative study

“…In such situations, robot controllers cannot purely rely on hard-coded models, even when precise CAD models of the robot are available [1]. The main reason for this is that kinematic controllers are not usually aware of the real dynamics of the robot.…”

“…They rely on decentralized joint level controllers which account for rigid-body dynamics and non-linear effects as unknown disturbances and thus do not act as pure integrators. Also, they assume that geometrical/mechanical models do not change over time, which does not fit in a real-life scenario where the robot may change its kinematics properties 1 , due to wear or malfunction. A possible strategy consists in letting the robot learn autonomously its own models, for example by interacting with the environment through its end-effectors (its hands in the simplest case), and exploiting a visual feedback to improve its accuracy while performing a task.…”

“…This work is supported by the French ANR program (ANR 2010 BLAN 0216 01 -http://macsi.isir.upmc.fr) and partially by the RTE company through its chair "Robotics Systems for field intervention in constrained environments" hold by Vincent Padois. 1 For example, elastic tendons actuating the joints may loosen in time, inducing small errors in the kinematics models which would be generally difficult to detect and automatically compensate for. In such contexts, Machine Learning (ML) techniques supersede classical approaches grounded on accurate exhaustive modeling [2], [3], [4].…”

Skill Learning for Intelligent Robot by Perception-Action Integration: A View from Hierarchical Temporal Memory

“…Skill learning autonomously through interactions with the environment is a crucial ability for intelligent robot, and it improves the flexibility and adaptiveness of robots [1]. Imitation learning is a primary method for implementation the skill learning [2].…”

Robot Skill Learning: From Reinforcement Learning to Evolution Strategies

“…To improve the flexibility and adaptiveness of robots -important requirements for robots operating in human environments -an increasing emphasis is being placed on robots that acquire skills autonomously through interactions with the environment [29]. State-of-the-art reinforcement learning (RL) algorithms such as PoWER and PI 2 are now able to learn very complex and high-dimensional robot skills [16,33].…”