Gain-field modulation mechanism in multimodal networks for spatial perception

Pitti, Alexandre; Blanchard, Arnaud; Cardinaux, Matthieu; Gaussier, Philippe

doi:10.1109/humanoids.2012.6651535

Cited by 12 publications

(12 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this work does not address the problem of coordinate transformations between different modalities in multisensory integration. Previous works done by the authors model the mechanism of gain-modulation found in parietal neurons for audio-visual and visuomotor coordinate transformations4344. In future works, one attempt will be to extend this model to coordinate tranformation of visuo-tactile and proprioceptive reference frames for simulating RHI with a robotic hand.…”

Section: Discussionmentioning

confidence: 98%

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Pitti¹,

Pugach²,

Gaussier³

et al. 2017

Sci Rep

Self Cite

View full text Add to dashboard Cite

Perceptual illusions across multiple modalities, such as the rubber-hand illusion, show how dynamic the brain is at adapting its body image and at determining what is part of it (the self) and what is not (others). Several research studies showed that redundancy and contingency among sensory signals are essential for perception of the illusion and that a lag of 200–300 ms is the critical limit of the brain to represent one’s own body. In an experimental setup with an artificial skin, we replicate the visuo-tactile illusion within artificial neural networks. Our model is composed of an associative map and a recurrent map of spiking neurons that learn to predict the contingent activity across the visuo-tactile signals. Depending on the temporal delay incidentally added between the visuo-tactile signals or the spatial distance of two distinct stimuli, the two maps detect contingency differently. Spiking neurons organized into complex networks and synchrony detection at different temporal interval can well explain multisensory integration regarding self-body.

show abstract

Section: Discussionmentioning

confidence: 98%

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Pitti¹,

Pugach²,

Gaussier³

et al. 2017

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…In previous works, we demonstrated how the mechanism of gain-field modulation can be applied for integrating audio-visual signals and proprioceptive feedback in a head-neck-eyes robotic device as it is for some parietal neurons (Pitti, Blanchard, Cardinaux, & Gaussier, 2012). In our studies, the gain-field neurons were successfully used to remap the location of one sound signal (in head-centered reference frame) into retina coordinates (in eyecentered reference frame).…”

Section: Introductionmentioning

confidence: 83%

Exploiting the gain-modulation mechanism in parieto-motor neurons: Application to visuomotor transformations and embodied simulation

et al. 2015

Self Cite

View full text Add to dashboard Cite

The so-called self-other correspondence problem in imitation demands to find the transformation that maps the motor dynamics of one partner to our own. This requires a general purpose sensorimotor mechanism that transforms an external fixation-point (partner's shoulder) reference frame to one's own body-centered reference frame. We propose that the mechanism of gain-modulation observed in parietal neurons may generally serve these types of transformations by binding the sensory signals across the modalities with radial basis functions (tensor products) on the one hand and by permitting the learning of contextual reference frames on the other hand. In a shoulder-elbow robotic experiment, gain-field neurons (GF) intertwine the visuo-motor variables so that their amplitude depends on them all. In situations of modification of the body-centered reference frame, the error detected in the visuo-motor mapping can serve then to learn the transformation between the robot's current sensorimotor space and the new one. These situations occur for instance when we turn the head on its axis (visual transformation), when we use a tool (body modification), or when we interact with a partner (embodied simulation). Our results defend the idea that the biologically-inspired mechanism of gain modulation found in parietal neurons can serve as a basic structure for achieving nonlinear mapping in spatial tasks as well as in cooperative and social functions.

show abstract

“…Their modeling corresponds to multiplicative Radial Basis Functions (RBFs) or sigma-pi networks [13], [14] to learn sensorimotor transformations. In image processing, these networks are known as gated networks, which have been recently re-investigated in [15], [16] for affine transformations and in developmental robotics [17], [18], [19] for multimodal integration. These multiplicative networks can serve to learn nonlinear transformations, which are common problems in robotics to compute direct mapping and inverse kinematics.…”

Section: Introductionmentioning

confidence: 99%

Visual Learning for Reaching and Body-Schema with Gain-Field Networks

Abrossimoff

Pitti

Gaussier

2018

2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

Self Cite

View full text Add to dashboard Cite

Perceiving our own body posture improves the way we move dynamically and reversely, motion coordination serves to learn better the position of our own body. Following this idea, we present a neural architecture toward reaching movements and body self-perception from a developmental perspective. Our framework is based on the neurobiological mechanism known as gain modulation in parietal neurons that is found to integrate the visual, motor and proprioceptive information through product-like processes. These multiplicative networks have interesting properties for learning nonlinear transformations such as the head-centered mapping in reaching tasks or the hand-centered mapping for a body-centered representation. In a simulation of a three-link arm, we perform experiments of nearby and far reach targets exploiting one or the other strategy. The later combination of the two networks generates autonomous control toward the target by processing the bodycentered spatial information and the preferred visual direction for the desired motor commands.

show abstract

Gain-field modulation mechanism in multimodal networks for spatial perception

Cited by 12 publications

References 24 publications

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Exploiting the gain-modulation mechanism in parieto-motor neurons: Application to visuomotor transformations and embodied simulation

Visual Learning for Reaching and Body-Schema with Gain-Field Networks

Contact Info

Product

Resources

About