Rapid Adaptation to Auditory-Visual Spatial Disparity

Abstract

The so-called ventriloquism aftereffect is a remarkable example of rapid adaptative changes in spatial localization caused by visual stimuli. After exposure to a consistent spatial disparity of auditory and visual stimuli, localization of sound sources is systematically shifted to correct for the deviation of the sound from visual positions during the previous adaptation period. In the present study, this aftereffect was induced by presenting, within 17 min, 1800 repetitive noise or pure-tone bursts in combination with synchronized, and 20° disparate flashing light spots, in total darkness. Post-adaptive sound localization, measured by a method of manual pointing, was significantly shifted 2.4° (noise), 3.1° (1 kHz tones), or 5.8° (4 kHz tones) compared with the pre-adaptation condition. There was no transfer across frequencies; that is, shifts in localization were insignificant when the frequencies used for adaptation and the post-adaptation localization test were different. It is hypothesized that these aftereffects may rely on shifts in neural representations of auditory space with respect to those of visual space, induced by intersensory spatial disparity, and may thus reflect a phenomenon of neural short-term plasticity.

Intersensory binding across space and time: A tutorial review

“…The critical examination is to use the same or different auditory frequencies in the exposure and test phases. The picture here is not entirely clear: Recanzone (1998) and Lewald (2002) reported that aftereffects did not transfer across frequencies of 750 and 3000 Hz Lewald, 2002;Recanzone, 1998), while Frissen and collaborators obtained transfer across even wider frequency differences of 400 and 6400 Hz (Frissen et al, 2003(Frissen et al, , 2005 Spatial ventriloquism and its aftereffect are also effective in improving spatial hearing in monaural conditions when interaural difference cues are not available. For example, Strelnikov, Rosito and Barone (2011) had observers wear an ear plug for 5 days, during which time they were trained in five 1-h sessions to localize monaural sounds.…”

“…For spatial ventriloquism, it consists of postexposure shifts in auditory localization toward the visual distractor (Bertelson, Frissen, Vroomen, & de Gelder, 2006;Canon, 1970;Frissen, Vroomen, de Gelder & Bertelson, 2003, 2005Lewald, 2002;Radeau, 1973Radeau, , 1992Radeau & Bertelson, 1969, 1974, 1978Recanzone, 1998;Zwiers, Van Opstal & Paige, 2003) and sometimes also in visual localization (e.g., Radeau, 1973;Radeau & Bertelson, 1974. A simple procedure for measuring aftereffects is depicted in Fig.…”

The biological basis of audition

“…This indicates that the representation of acoustic space has somehow been altered by the disparate visual experience. This can occur when the auditory and visual stimuli are displaced by a small amount and are consistently perceived to be from the same location (Recanzone 1998) or when the spatial disparity is quite large (Lewald 2002). Under either condition, the effect does not transfer across frequencies, i.e., training at one frequency does not influence the spatial representation at other frequencies.…”

“…An intriguing aspect of the ventriloquism effect is that it can be long lasting (Lewald 2002, Radeau & Bertelson 1974, Recanzone 1998, Woods & Recanzone 2004. If a subject is presented with auditory and visual stimuli with a consistent spatial disparity for a period of tens of minutes, the subject's percept of acoustic space will be shifted in the direction of the visual stimulus when tested in complete darkness.…”

“…The frequency range over which this integration occurs remains to be explored. Recanzone (1998) used 750 Hz and 3000 Hz stimuli, and Lewald (2002) used 1000 Hz and 4000 Hz, so potentially a frequency difference of less than two octaves could show a transference effect. The duration of the effect also remains untested.…”