Auditory Motion Capturing Ambiguous Visual Motion

Abstract

In this study, it is demonstrated that moving sounds have an effect on the direction in which one sees visual stimuli move. During the main experiment sounds were presented consecutively at four speaker locations inducing left or rightward auditory apparent motion. On the path of auditory apparent motion, visual apparent motion stimuli were presented with a high degree of directional ambiguity. The main outcome of this experiment is that our participants perceived visual apparent motion stimuli that were ambiguous (equally likely to be perceived as moving left or rightward) more often as moving in the same direction than in the opposite direction of auditory apparent motion. During the control experiment we replicated this finding and found no effect of sound motion direction on eye movements. This indicates that auditory motion can capture our visual motion percept when visual motion direction is insufficiently determinate without affecting eye movements.

Auditory Motion Elicits a Visual Motion Aftereffect

“…This finding is in contrast to previous findings demonstrating that rightward moving auditory stimuli could capture ambiguous visual motion stimuli, but that leftward moving stimuli did not (Alink et al, 2012). In our study, we made use of pre-recorded leftward moving auditory motion stimuli that were digitally reversed so that they were perceived as moving rightward.…”

“…This interpretation of our finding is consistent with the maximum likelihood estimation view of multisensory perception (Ernst and Banks, 2002; De Gelder and Bertelson, 2003; Ernst and Bülthoff, 2004) as well as findings in research on multisensory integration at large demonstrating that auditory stimuli can have a profound influence on visual perception if the auditory stimuli are sufficiently reliable (Driver and Spence, 2000; Shimojo and Shams, 2001; Ghazanfar and Schroeder, 2006; Alink et al, 2012). In the case of motion perception in particular, for example, it has been demonstrated that auditory spatial information provided by alternating sound locations can cause a static visual stimulus to be perceived as moving in an illusion referred to as the sound-induced visual motion illusion (Hidaka et al, 2011b) and that auditory motion stimuli can capture ambiguous visual motion and change the perceived apparent motion of visual stimuli in the same direction as the moving auditory stimuli (Alink et al, 2012).…”

“…Although, this procedure held the timing and quality of the auditory motion stimuli constant, we cannot rule out that this could have led to some unforeseen differences between the leftward moving and rightward moving stimuli that could have somehow affected the vMAE. However, this interpretation is incompatible with the opposite asymmetry found in Alink et al's (2012) study in which an identical presentation protocol was used to present the leftward and rightward auditory motion stimuli from external speakers but only an effect of rightward auditory motion on visual motion perception was observed. One alternative possibility that reconciles the findings of Alink et al (2012) and ours is that that the influence of auditory stimuli on visual motion perception is anisotropic (i.e., not equal for movement in both directions) as both findings indicate an increase in the perception that ambiguous visual motion stimuli are moving rightward but not that they are moving leftward.…”

“…However, this interpretation is incompatible with the opposite asymmetry found in Alink et al's (2012) study in which an identical presentation protocol was used to present the leftward and rightward auditory motion stimuli from external speakers but only an effect of rightward auditory motion on visual motion perception was observed. One alternative possibility that reconciles the findings of Alink et al (2012) and ours is that that the influence of auditory stimuli on visual motion perception is anisotropic (i.e., not equal for movement in both directions) as both findings indicate an increase in the perception that ambiguous visual motion stimuli are moving rightward but not that they are moving leftward. However, this interpretation would be at odds with the existing literature on visual motion perception and visual motion aftereffects which largely suggest that the motion processing mechanisms of centrally-viewed motion stimuli in the horizontal plane are isotropic (Levinson and Sekuler, 1975; Ball and Sekuler, 1979; Mather, 1980; Raymond, 1993).…”

“…In the case of motion perception in particular, for example, it has been demonstrated that auditory spatial information provided by alternating sound locations can cause a static visual stimulus to be perceived as moving in an illusion referred to as the sound-induced visual motion illusion (Hidaka et al, 2011b) and that auditory motion stimuli can capture ambiguous visual motion and change the perceived apparent motion of visual stimuli in the same direction as the moving auditory stimuli (Alink et al, 2012). Furthermore, functional magnetic resonance imagining (fMRI) experiments have revealed that visual motion area MT/V5 responded to auditory motion in blind patients (Saenz et al, 2008), that visual motion capture of auditory motion is reflected in increased activity in area MT/V5 (Alink et al, 2008), and that concordant audiovisual motion stimuli lead to enhanced activity in area MT/V5 compared unimodal motion stimuli or discordant audiovisual motion stimuli (Scheef et al, 2009).…”