An Autonomous Developmental Cognitive Architecture Based on Incremental Associative Neural Network With Dynamic Audiovisual Fusion

Abstract

Developing cognition is difficult to achieve yet crucial for robots. Infants can gradually improve their cognition through parental guidance and self-exploration. However, conventional learning methods for robots often focus on a single modality and train a pre-defined model by large datasets in an offline way. In this paper, we propose a hierarchical autonomous cognitive architecture for robots to learn object concepts online by interacting with humans. Two pathways for audiovisual information are devised. Each pathway has three layers based on the self-organizing incremental neural networks. Visual features and names of objects are incrementally learned and self-organized in an unsupervised way in sample layers, respectively, in which we propose a dynamically adjustable similarity threshold strategy to allow the network itself to control cluster rather than using a pre-defined threshold. Two symbol layers abstract the cluster results from the corresponding sample layer to form concise symbols and transmit them to an associative layer. An associative relationship between two modalities can be built in real time by binding activated visual and auditory symbols simultaneously in the associative layer. In this layer, a top-down response strategy is proposed to let robots autonomously recall another associative modality, solve conflicting associative relationships, and adjust learned knowledge from the top down. The experimental results on two objects datasets and a real task show that our architecture is efficient to learn and associate object view and name in an online way. What is more, the robot can autonomously improve its cognitive level by utilizing its own experience without enquiring with humans. INDEX TERMS Cognitive development, concept online learning, self-organizing incremental neural network, object recognition, audiovisual integration.

Biologically Inspired Self-Organizing Computational Model to Mimic Infant Learning

“…Employing self-organizing computational models to assistive robots by taking inspiration from nature, enacts new perspectives that result in human-like decision-making capabilities [38,39]. By eliminating humans from the learning process, robots can directly learn from the data available.…”

“…Further, it helps in planning the secure path to navigate the environment. Huang et al [39,44,45] proposed a dynamic threshold self-organizing incremental neural network (DT-SOINN) based on hierarchical cognitive architecture for assistive robots. The proposed architecture combined auditory and visual subsystems and learned to form an association between them.…”

Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion

“…We aim to create a new multimodal topology over which new dynamic properties could be applied, and selforganization offers solutions for a much lower cost [15]- [23]. SOM and their derivatives have long been used as models of multimodal fusion, but the ways modalities are combined can be very diverse.…”

“…In the second, unimodal maps link to a new multimodal SOM [18], [19] or NG [20] that combines all information. Additional layers of SOM can also be considered to create a hierarchical flow of information [21]- [23]. Additionally, models can be made more adaptive to timedependant tasks with the help of "growing when required" maps [22], [23], an alternative to GNG designed for dynamic input distributions [24].…”

“…Additional layers of SOM can also be considered to create a hierarchical flow of information [21]- [23]. Additionally, models can be made more adaptive to timedependant tasks with the help of "growing when required" maps [22], [23], an alternative to GNG designed for dynamic input distributions [24]. Some of these models have already been proof-tested for visual, auditory and/or proprioceptive modalities on hardware setups [21], [23] and robots [17], [19].…”