Change of Temporal-Order Judgment of Sounds during Long-Lasting Exposure to Large-Field Visual Motion

Abstract

The perceived temporal order of external successive events does not always follow their physical temporal order. We examined the contribution of self-motion mechanisms in the perception of temporal order in the auditory modality. We measured perceptual biases in the judgment of the temporal order of two short sounds presented successively, while participants experienced visually induced self-motion (yaw-axis circular vection) elicited by viewing long-lasting large-field visual motion. In experiment 1, a pair of white-noise patterns was presented to participants at various stimulus-onset asynchronies through headphones, while they experienced visually induced self-motion. Perceived temporal order of auditory events was modulated by the direction of the visual motion (or self-motion). Specifically, the sound presented to the ear in the direction opposite to the visual motion (ie heading direction) was perceived prior to the sound presented to the ear in the same direction. Experiments 2A and 2B were designed to reduce the contributions of decisional and/or response processes. In experiment 2A, the directional cueing of the background (left or right) and the response dimension (high pitch or low pitch) were not spatially associated. In experiment 2B, participants were additionally asked to report which of the two sounds was perceived 'second'. Almost the same results as in experiment 1 were observed, suggesting that the change in temporal order of auditory events during large-field visual motion reflects a change in perceptual processing. Experiment 3 showed that the biases in the temporal-order judgments of auditory events were caused by concurrent actual self-motion with a rotatory chair. In experiment 4, using a small display, we showed that 'pure' long exposure to visual motion without the sensation of self-motion was not responsible for this phenomenon. These results are consistent with previous studies reporting a change in the perceived temporal order of visual or tactile events depending on the direction of self-motion. Hence, large-field induced (ie optic flow) self-motion can affect the temporal order of successive external events across various modalities.

Effect of Visual Stimuli on Temporal Order Judgments of a Sequence of Pure Tones

“…Our experiments revealed that the temporal perception of auditory events affected visual spatial information even if it was not a sufficiently large-field visual stimulus to be perceived as self-motion, whereas Teramoto et al ( 2008 ) reported that auditory TOJs were not affected by visual stimulus when their participants did not perceive self-motion. Thus, large-field visual stimuli are not always necessary to affect the temporal perception of auditory stimuli.…”

“…Our aim in this paper is to present an example where visual spatial information can affect the TOJ tasks of auditory stimuli that are spatially uninformative unlike those in Teramoto et al ( 2008 ). The auditory stimuli in our experiments were a sequence of pure tones that were spatially uninformative, since the same auditory stimuli were presented to both ears.…”

“…Teramoto, Watanabe, Umemura, and Kita ( 2008 ) recently reported that self-motion induced by a large field is able to affect the perception of the temporal order of tones. They used a rotating virtual cylinder with random dots as the visual stimuli and two successive sounds as the auditory stimuli; the first auditory stimulus was presented to the right ear and the second was presented to the left ear.…”

Multisensory Integration With a Head-Mounted Display: Background Visual Motion and Sound Motion

“…Väljamäe (2009) demonstrates that moving sounds may enhance vection, an illusory feeling of self-motion induced by a constantly moving visual environment. Teramoto, Watanabe, and Umemura (2008) and Teramoto, Watanabe, Umemura, Matsuoka, and Kita (2004) show that in conditions of vection, people misjudge the temporal order of visual events, of sounds, and of tactile stimuli, but they have not examined the effect of vection on multisensory integration across any of these modalities. Väljamäe speculates that the directional congruence between a moving sound and moving visual environment may be crucial for the perception of motion but that any interaction between vision, audition, and other sensory modalities that could influence basic perception is poorly understood.…”

“…As noted in the Introduction, Väljamäe (2009) highlights the need for future research to understand more fully how optokinetic stimulation, vection, and auditory perception may interact. As also noted, Teramoto et al (2004Teramoto et al ( , 2008 have shown that under conditions of self-motion, people misjudge the temporal order of visual events, of sounds, and of tactile stimuli, but they have not examined the effect of self-motion on multisensory integration across any of these modalities. Further research is needed with procedures such as theirs to corroborate our findings and to examine the role of eye reflexes in such phenomena.…”