A Self-Organized Artificial Neural Network Architecture for Sensory Integration with Applications to Letter-Phoneme Integration

Abstract

The multimodal self-organizing network (MMSON), an artificial neural network architecture carrying out sensory integration, is presented here. The architecture is designed using neurophysiological findings and imaging studies that pertain to sensory integration and consists of interconnected lattices of artificial neurons. In this artificial neural architecture, the degree of recognition of stimuli, that is, the perceived reliability of stimuli in the various subnetworks, is included in the computation. The MMSON's behavior is compared to aspects of brain function that deal with sensory integration. According to human behavioral studies, integration of signals from sensory receptors of different modalities enhances perception of objects and events and also reduces time to detection. In neocortex, integration takes place in bimodal and multimodal association areas and result, not only in feedback-mediated enhanced unimodal perception and shortened reaction time, but also in robust bimodal or multimodal percepts. Simulation data from the presented artificial neural network architecture show that it replicates these important psychological and neuroscientific characteristics of sensory integration.

Multimodal space representation driven by self-evaluation of predictability

“…It is also conceptually very close to the predictive coding model of hierarchical visual processing [17], [18], [19]. This focus on predictability and on generic multimodal processing are the two main points that distinguish our work from other multimodal self-organizing maps models [20], [21], [22].…”

A generative-discriminative learning model for noisy information fusion

“…Some recent papers used SOM or SOM-like bio-imitating architectures in order to reproduce individual behaviours or biological phenomena by imitating, more or less clumsily, hierarchical layered interconnected cortical areas, especially superior colliculus ( [7] and [8] which place emphasis on the positive impact of a non-linear transfer function applied to neural maps and [9], [10] and [11] which deal with imitating SC multisensory integration). Several studies aim at designing models inspired by recent neurophysiological findings without fully copying nature architectures or processes, focusing on well-defined applications like information retrieval [12], visual categorisation [13], data visualisation [14] or audio-visual multimodal integration [15].…”

Consensus Driven Self-Organization: Towards Non Hierarchical Multi-Map Architectures

“…In the end, quite a few works address modular and topographical self-organization. Let us mention [7] which is an algorithmical approach oriented toward letter-phoneme integration as well as A-SOM [8] which introduces associative SOMs. The present work is in line with these two, with a higher stress on computational homogeneity between the modules and scalability for architectures with many modules.…”