Learn to wipe: A case study of structural bootstrapping from sensorimotor experience

Abstract

In this paper, we address the question of generative knowledge construction from sensorimotor experience, which is acquired by exploration. We show how actions and their effects on objects, together with perceptual representations of the objects, are used to build generative models which then can be used in internal simulation to predict the outcome of actions. Specifically, the paper presents an experiential cycle for learning association between object properties (softness and height) and action parameters for the wiping task and building generative models from sensorimotor experience resulting from wiping experiments. Object and action are linked to the observed effect to generate training data for learning a nonparametric continuous model using Support Vector Regression. In subsequent iterations, this model is grounded and used to make predictions on the expected effects for novel objects which can be used to constrain the parameter exploration. The cycle and skills have been implemented on the humanoid platform ARMAR-IIIb. Experiments with set of wiping objects differing in softness and height demonstrate efficient learning and adaptation behavior of action of wiping.

Staged Development of Robot Skills: Behavior Formation, Affordance Learning and Imitation with Motionese

“…In the future, we plan to study how the proposed approach can scale up to support complex tasks that include manipulation and interaction of multiple objects with the increasing multiplicity and diversity of real-world datasets [89]. Mechanisms like alternating skill development [88], structural bootstrapping [90], [91], [92], transfer learning [93], and intrinsic motivation [94], [85] should be utilized to cope with the complexity of the learning in such large sensorimotor spaces.…”

Structural Bootstrapping—A Novel, Generative Mechanism for Faster and More Efficient Acquisition of Action-Knowledge

“…• wipe a surface with a sponge [25]- [27]. Furthermore the robot has a certain type of object memory where it has stored a set of objects together with their roles, called the Repository of objects with attributes and roles (ROAR).…”

“…• The robot can recognize known objects (tools, ingredients, batter) using computer vision [28], [29], [30]. • The robot can explore unknown object haptically [31] and extract object features such as deformability and softness [32], [33], [25] C…”

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

“…It addresses mechanisms relying on prior knowledge, sensorimotor experience, and inference that can be implemented in robotic systems and employed to speed up learning and problem solving in new environments. Earlier experiments demonstrate how structural bootstrapping can be applied at different levels of a robotic architecture including a sensorimotor level, a symbolto-signal mediator level, and a planning level [2], [3].…”