The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity

Abstract

Information about an event takes different amounts of time to be processed depending on which sensory system the event activates. However, despite the variations in processing time for lights and sounds, the point of subjective simultaneity (PSS) for briefly presented audio/visual stimuli is usually close to true simultaneity. Here we confirm that the simultaneity constancy mechanism that achieves this for audio/visual stimulus pairs is adaptable, and extend the investigation to other multimodal combinations. We measured the PSS and just noticeable differences (JNDs) for temporal order judgements for three stimulus combinations (sound/light, sound/touch, and light/touch) before and after repeated exposure to each one of these pairs presented with a 100 ms asynchrony (i.e., nine adapt-test combinations). Only the perception of simultaneity of the sound/light pair was affected by our exposure regime: the PSS shifted after exposure to either a temporally staggered sound/light or light/touch pair, and the JND decreased following exposure to a sound/touch pair. No changes were found in the PSSs or JNDs of sound/touch or light/touch pairs following exposure to any of the three time-staggered combinations. Participants' reaction times (RT) to the three stimuli were also tested before and after each adaptation exposure. In general, exposure did not affect attention or processing time; the only change in RTs (of the 9 tested) was an increased RT for light following exposure to a sound/light pair with light leading. We suggest that the neural correlates of multisensory sound/light processing are resynchronised by a separate, more flexible simultaneity constancy mechanism than the light/touch or the sound/touch simultaneity processing systems.

Adaptation to audiovisual asynchrony modulates the speeded detection of sound

“…RTs, in our experiment, increased or decreased as the exposure to an AV or VA asynchrony, respectively, augmented. Whether or not performance will be optimal as a result of this temporal readjustment † Harrar et al's study (3) represents an interesting attempt to see the effects of perceiving audiovisual asynchrony on the detection of visual or auditory stimuli. However, while the methodology used by these authors was designed to investigate any possible effects of attention, it did not allow them to see specific modulations of the RTs to visual and auditory stimuli due to temporal recalibration.…”

Perception of intersensory synchrony: A tutorial review

“…They exposed participants for 5 min to audio-visual pairs with a fixed time lag (250 msec, light first) but did not obtain shifts in the PSSs for touch-light pairs. In an extension of this topic (Harrar & Harris, 2008), observers were exposed for 5 min to ~100-msec lags of light-first stimuli for the audio-visual case, and touch-first stimuli for the auditory-tactile and visual-tactile case. Participants were tested on each of these pairs before and after exposure.…”

Shifts of criteria or neural timing? The assumptions underlying timing perception studies

“…Temporal recalibration has been assessed with a variety of tasks, including simple reaction times (Di Luca, Machulla & Ernst, 2009;Harrar & Harris, 2008;Navarra, Hartcher-O'Brien, Piazza & Spence, 2009) and the perception of ambiguous visual stimuli in bimodal displays (Fujisaki et al, 2004). However, it is typically measured as a shift in the PSS derived from TOJs or SJs between conditions with different adapting asynchronies.…”

“…Against them must be weighed the following consideration at least: Recalibration can be observed using tasks other than simultaneity judgements, including temporal order judgements, simple reaction times, and the stream-bounce illusion (e.g. Di Luca et al, 2009;Fujisaki et al, 2004;Hanson, Heron & Whitaker, 2008;Harrar & Harris, 2008;Navarra et al, 2009). However, all of these tasks also involve criterion settings, even the apparently implicit judgements in the multimodal bounce-stream illusion (Grove et al, 2009).…”