Vision usually provides the most accurate and reliable information about the location of objects in our environment, and thus serves as a reference for recalibrating auditory spatial maps. Recent studies have shown that recalibration does not require accumulated evidence of cross-modal mismatch to be triggered, but occurs as soon as after one single exposure. Here we tested whether instantaneous recalibration and recalibration based on accumulated evidence represent the same underlying learning mechanism or involve distinct neural systems. Participants had to localize two sounds, a low- and a high-frequency tone, which were paired with opposite directions of audiovisual spatial mismatch (leftward vs. rightward). In accordance with the cumulative stimulus history, localization in unimodal auditory trials was shifted in opposite directions for the two sound frequencies. On a trial-by-trial basis, however, frequency-specific recalibration was reduced when preceded by an audiovisual stimulus with a different sound frequency and direction of spatial mismatch. Thus, the immediate past invoked an instantaneous frequency-invariant recalibration, while the cumulative past invoked changes in frequency-specific spatial maps. These findings suggest that distinct recalibration mechanisms operating at different timescales jointly determine sound localization behavior.
In the ventriloquism aftereffect, brief exposure to a consistent spatial disparity between auditory and visual stimuli leads to a subsequent shift in subjective sound localization toward the positions of the visual stimuli. Such rapid adaptive changes probably play an important role in maintaining the coherence of spatial representations across the various sensory systems. In the research reported here, we used event-related potentials (ERPs) to identify the stage in the auditory processing stream that is modulated by audiovisual discrepancy training. Both before and after exposure to synchronous audiovisual stimuli that had a constant spatial disparity of 15°, participants reported the perceived location of brief auditory stimuli that were presented from central and lateral locations. In conjunction with a sound localization shift in the direction of the visual stimuli (the behavioral ventriloquism aftereffect), auditory ERPs as early as 100 ms poststimulus (N100) were systematically modulated by the disparity training. These results suggest that cross-modal learning was mediated by a relatively early stage in the auditory cortical processing stream.
Highlights d Multisensory integration develops prior to unsupervised crossmodal recalibration d Multisensory integration and crossmodal learning are dissociable processes d Crossmodal recalibration in humans emerges first for short timescales d Crossmodal recalibration requiring accumulation of evidence develops last
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.