Learning tactile skills through curious exploration

Abstract

We present curiosity-driven, autonomous acquisition of tactile exploratory skills on a biomimetic robot finger equipped with an array of microelectromechanical touch sensors. Instead of building tailored algorithms for solving a specific tactile task, we employ a more general curiosity-driven reinforcement learning approach that autonomously learns a set of motor skills in absence of an explicit teacher signal. In this approach, the acquisition of skills is driven by the information content of the sensory input signals relative to a learner that aims at representing sensory inputs using fewer and fewer computational resources. We show that, from initially random exploration of its environment, the robotic system autonomously develops a small set of basic motor skills that lead to different kinds of tactile input. Next, the system learns how to exploit the learned motor skills to solve supervised texture classification tasks. Our approach demonstrates the feasibility of autonomous acquisition of tactile skills on physical robotic platforms through curiosity-driven reinforcement learning, overcomes typical difficulties of engineered solutions for active tactile exploration and underactuated control, and provides a basis for studying developmental learning through intrinsic motivation in robots.

The feeling of grip: novelty, error dynamics, and the predictive brain

“…These ideas about learning progress have shown promise in developmental robotics, allowing robots implementing these routines to "efficiently learn repertoires of skills in high dimensions and under strong time constraints and to avoid unfruitful activities that are either well learnt and trivial, or which are random and unlearnable (Pape et al 2012;Ngo et al 2012;Baranes and Oudeyer 2013;Nguyen and Oudeyer 2013)" (Gottlieb et al 2013: p. 9). There are good reasons to suspect a similar approach to learning takes place in humans.…”

Feeling the Shape: Active Exploration Behaviors for Object Recognition With a Robotic Hand

“…Cognitive architectures for active exploration, learning, information-seeking and attention with computational agents, were proposed with multiple intrinsic motivation models, e.g., information gain, predictive novelty, distributional surprise, distributional familiarity [43], [44], [45]. Learning of robot skills was investigated with intrinsically motivated movements, allowing a robot to explore interesting stimuli while maximising visual and tactile perception [46], [47]. Active exploration and intrinsic motivation allowed a robot to learn inverse models and motor primitives [48], [49].…”

Exploration Behaviors, Body Representations, and Simulation Processes for the Development of Cognition in Artificial Agents

“…Ngo et al (2013), for example, proposed a system that generates goals based on the confidence in its predictions about how the environment reacts to its actions; when the confidence on a prediction is low, the environmental configuration that generated such an event becomes a goal. Pape et al (2012) presented a similar curiositydriven exploration behavior in the context of tactile skills learning, which allowed the robotic system to autonomously develop a small set of basic motor skills that lead to different kinds of tactile input, and to learn how to exploit the learned motor skills to solve texture classification tasks. Jauffret et al (2013) presented a neural architecture based on an online novelty detection algorithm that is able to self-evaluate sensory-motor strategies.…”