Abstract. We present a vision system for human-machine interaction that relies on a small wearable camera which can be mounted to common glasses. The camera views the area in front of the user, especially the hands. To evaluate hand movements for pointing gestures to objects and to recognise object reference, an approach relying on the integration of bottom-up generated feature maps and top-down propagated recognition results is introduced. In this vision system, modules for context free focus of attention work in parallel to a recognition system for hand gestures. In contrast to other approaches, the fusion of the two branches is not on the symbolic but on the sub-symbolic level by use of attention maps. This method is plausible from a cognitive point of view and facilitates the integration of entirely different modalities.
A major goal for the realization of a new generation of intelligent robots is the capability of instructing work tasks by interactive demonstration. To make such a process e&ient and convenient for the human user requires that both the robot and the user can establish and maintain a common focus of attention. We describe a hybrid architecture that combines neural networks and finite state machines into a flexible framework for controlling the behaviour of a vision based robot called GRAVIS-robot (Gestural Recognition Active VIsion System robot). It consists of a binocular camera head, a 6 DOF robot arm and a 9 DOF multifinsered hand. We focus primarily on non-verbal communication based on gestural commands of a human instructor which will at a later stage be complemented by spoken instructions.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.