Guiding attention for grasping tasks by gestural instruction: the GRAVIS-robot architecture

Abstract

A major goal for the realization of a new generation of intelligent robots is the capability of instructing work tasks by interactive demonstration. To make such a process e&ient and convenient for the human user requires that both the robot and the user can establish and maintain a common focus of attention. We describe a hybrid architecture that combines neural networks and finite state machines into a flexible framework for controlling the behaviour of a vision based robot called GRAVIS-robot (Gestural Recognition Active VIsion System robot). It consists of a binocular camera head, a 6 DOF robot arm and a 9 DOF multifinsered hand. We focus primarily on non-verbal communication based on gestural commands of a human instructor which will at a later stage be complemented by spoken instructions.

Focus-of-attention from local color symmetries

“…Images were acquired from the "Baufix 1 -scenario" of the collaborative research project "Situated Artificial Communicators" at Bielefeld University, Germany (see, e.g., [54], [7], [5], [61]). The objects are wooden toy pieces of the Baufix 1 -system.…”

“…In the Baufix-scenario of the collaborative research project SFB 360 at Bielefeld University, detection of toy pieces within assemblies is required in the context of a human-machine interaction scenario, where a robot assembles the toy pieces guided by hand gestures and speech from a human instructor. For details see, e.g., [54], [7], [5], [61]. As many of the toy pieces have similar gray values, color is necessary: Fig.…”

Integrating Inhomogeneous Processing and Proto-object Formation in a Computational Model of Visual Attention

“…With the advent of increasingly cognitive robots using active vision, computational models for guidance of their visual attention become a standard component in their control architectures [1,6,12,16]. Many of these models are driven by the ambition to realize human-like perception and action and rely on bottom-up computation of features.…”

Platform portable anthropomorphic grasping with the bielefeld 20-DOF shadow and 9-DOF TUM hand

“…Consequently, we have previously proposed to enable imitation grasping [18] in the context of a long-term research project [19] aiming at the realization of a robot system that is instructable by speech and gesture, has visual capabilities, attentive behavior, and can execute grasping actions [20], [21] (see Fig. 1).…”