The ventriloquist in periphery: Impact of eccentricity-related reliability on audio-visual localization

Abstract

The relative reliability of separate sensory estimates influences the way they are merged into a unified percept. We investigated how eccentricity-related changes in reliability of auditory and visual stimuli influence their integration across the entire frontal space. First, we surprisingly found that despite a strong decrease in auditory and visual unisensory localization abilities in periphery, the redundancy gain resulting from the congruent presentation of audio-visual targets was not affected by stimuli eccentricity. This result therefore contrasts with the common prediction that a reduction in sensory reliability necessarily induces an enhanced integrative gain. Second, we demonstrate that the visual capture of sounds observed with spatially incongruent audio-visual targets (ventriloquist effect) steadily decreases with eccentricity, paralleling a lowering of the relative reliability of unimodal visual over unimodal auditory stimuli in periphery. Moreover, at all eccentricities, the ventriloquist effect positively correlated with a weighted combination of the spatial resolution obtained in unisensory conditions. These findings support and extend the view that the localization of audio-visual stimuli relies on an optimal combination of auditory and visual information according to their respective spatial reliability. All together, these results evidence that the external spatial coordinates of multisensory events relative to an observer's body (e.g., eyes' or head's position) influence how this information is merged, and therefore determine the perceptual outcome.

Representation of central and peripheral vision in the primate cerebral cortex: Insights from studies of the marmoset brain

“…Electrophysiological and human psychophysics studies have taken the first steps in the investigation of cross-modal interactions of visual and auditory stimuli in the peripheral visual field (Wang et al, 2008;Charbonneau et al, 2013;Gleiss and Kayser, 2013). The effect of attention is another direction that has produced promising results (Jack et al, 2006;Cate et al, 2009).…”

Multisensory stimuli improve relative localisation judgments compared to unisensory auditory or visual stimuli

“…Both auditory (Mills, 1958; Makous and Middlebrooks, 1990; Charbonneau et al , 2013; Wood and Bizley, 2015; Carlile et al , 2016) and visual localisation acuity declines with eccentricity (Mateeff and Gourevich, 1984; Perrott et al , 1993; Charbonneau et al , 2013). Few studies have attempted to directly compare spatial acuity for auditory and visual stimuli throughout the visual field and focus instead on the spatial capture observed when spatially separated auditory-visual signals are presented (Howard and Templeton, 1966; Bertelson and Radeau, 1981).…”

Multisensory cue combination after sensory loss: Audio-visual localization in patients with progressive retinal disease.

“…However, Charbonneau, Veronneau, Boudrias-Fournier, Lepore, and Collignon (2013) found that the visual capture of spatially misaligned auditory information in human adults declines with eccentricity, suggesting that adults do reduce their reliance on vision in audio-visual peripheral spatial decisions.…”