Neural Architectures for Robot Intelligence

Abstract

SYNOPSISWe argue that direct experimental approaches to elucidate the architecture of higher brains may benefit from insights gained from exploring the possibilities and limits of artificial control architectures for robot systems. We present some of our recent work that has been motivated by that view and that is centered around the study of various aspects of hand actions since these are intimately linked with many higher cognitive abilities. As examples, we report on the development of a modular system for the recognition of continuous hand postures based on neural nets, the use of vision and tactile sensing for guiding prehensile movements of a multifingered hand, and the recognition and use of hand gestures for robot teaching.Regarding the issue of learning, we propose to view real-world learning from the perspective of data-mining and to focus more strongly on the imitation of observed actions instead of purely reinforcement-based exploration. As a concrete example of such an effort we report on the status of an ongoing project in our laboratory in which a robot equipped with an attention system with a neurally inspired architecture is taught actions by using hand gestures in conjunction with speech commands. We point out some of the lessons learnt from this system, and discuss how systems of this kind can contribute to the study of issues at the junction between natural and artificial cognitive systems.

A Dual Interaction Perspective for Robot Cognition: Grasping as a “Rosetta Stone”

“…Based on these very encouraging results, we are currently porting the capabilities of a previously developed system with a more limited robot manipulator, but coupled with speech-understanding and binocular vision capabilities [2,6,7], to the new architecture. Developed over a time horizon of several years, this previous system had reached the limits of its extensibility and maintainability.…”

“…A few examples of such systems acting in a robotics or pure vision domain can be found in [2,3,4,5]. Some of our own earlier work along these lines is summarised in [6,7]. Experience from this work led to the dual interaction perspective described above and to ideas for its implementation in a compact and highly manageable architecture employing hierarchical state machines (HSMs).…”

Situated robot learning for multi-modal instruction and imitation of grasping

“…In this framework, very heterogeneous components can be accommodated as separate and parallel processes, distributed over several workstations and communicating mainly via message passing supported by DACS (some modules also use more sophisticated communication facilities of the DACS package). In this way, we have been able to integrate a large number of modules, which use different programming languages (C, C++, Tcl/Tk, Neo/NST), various visualization tools, and a variety of processing paradigms ranging from a neurally inspired attention system to statistical and declarative methods for inference and knowledge representation [25,29].…”

“…This combination of observation and reinforcement learning appears very flexible: the neighborhood can be chosen small where highly reliable observations are available, whereas more exploration may be needed where poor data are given. A typical candidate for the application of this approach in our scenario would be the initialization of a grasping sequence with respect to the approach direction and hand pre-shape based on visual observation of a human hand, which can be obtained by earlier developed hand and fingertip recognition methods (see Section 5 and [27,29]). …”

Adaptive scene dependent filters for segmentation and online learning of visual objects

“…In most systems recognition performance thus depends crucially on assuming uncluttered background, homogeneous coloring of foreground objects, or predefined object classes to facilitate segmentation of objects against their surrounding. One such approach to reach the semantic level based on an attention system is to search for known objects at the current fixation point with a holistic object classification system (5) and to store objects recognized in a symbolic memory (6; 7). But the system needs a large amount of training images from different views, and the object classification itself has to be trained offline and beforehand.…”