Capture of Intermodal Visual/Tactile Apparent Motion by Moving and Static Sound

Abstract

Apparent motion can occur within a particular modality or between modalities, in which a visual or tactile stimulus at one location is perceived as moving towards the location of the subsequent tactile or visual stimulus. Intramodal apparent motion has been shown to be affected or 'captured' by information from another, task-irrelevant modality, as in spatial or temporal ventriloquism. Here we investigate whether and how intermodal apparent motion is affected by motion direction cues or temporal interval information from a third modality. We demonstrated that both moving and asynchronous static sounds can capture intermodal (visual-tactile and tactile-visual) apparent motion; moreover, while the auditory direction cues have less impact upon the perception of intramodal visual apparent motion than upon the perception of intramodal tactile or intermodal visual/tactile apparent motion, the auditory temporal information has equivalent impacts upon both intramodal and intermodal apparent motion. These findings suggest intermodal apparent motion is susceptible to the influence of dynamic or static auditory information in similar ways as intramodal visual or tactile apparent motion.

Grades of Multisensory Awareness

“…For example, with an appropriate time interval between a visual stimulus at one location and a tactile stimulus at another location, the participants would perceive some kind of motion stream from the first to the second location. In this kind of intermodal apparent motion, the motion stream is composed of stimuli from two different modalities (Chen and Zhou, , pp. 369, 371).…”

“…369, 371). Chen and Zhou (2011) and Jiang and Chen (2013) report that auditory and visuo-tactile apparent motion influence each other.…”

Intersensory binding across space and time: A tutorial review

“…Spatial ventriloquism can also be found with dynamic stimuli. In apparent motion, visual motion direction can attract the perceived direction of auditory motion (Kitajima & Yamashita, 1999;Mateeff, Hohnsbein & Noack, 1985;Soto-Faraco, Lyons, Gazzaniga, Spence & Kingstone, 2002;Soto-Faraco, Spence & Kingstone, 2004a, b, 2005Stekelenburg & Vroomen, 2009), and auditory motion can attract visual motion (Alais & Burr, 2004a;Chen & Zhou, 2011;Meyer & Wuerger, 2001;Wuerger, Hofbauer & Meyer, 2003).…”

“…Observers reported that a constant flicker rate altered when the flutter changed, whereas the reverse effect (visual flicker altering auditory Role of attention -Direction of endogenous/exogenous shift of attention and shift in sound location can be dissociated -Sounds preferably segregated with sharp onsets -But arrows and gaze can induce shift sound location as well -Attention to the audiovisual timing relation increases aftereffect -Dual task with focused attention does not decrease the aftereffect Audiovisual congruence -Phonetic congruency in speech: no effect -Gender-matched speech: more fusion -Face orientation: no effect--Pitch/size congruence: more fusion for congruent pairs -Speech/nonspeech mode with sine wave speech: no effect -Nonspeech like musical instruments: no effect of audiovisual congruency -Pitch/size congruence: greater effect for congruent pairs flutter) could not be observed. In more recent years, temporal ventriloquism has been demonstrated in a number of other paradigms: Besides auditory driving (Bresciani & Ernst, 2007;Gebhard & Mowbray, 1959;Recanzone, 2003;Shipley, 1964;Welch, DuttonHurt & Warren, 1986), or a variant of this called the double-flash illusion (Shams, Kamitani & Shimojo, 2000), researchers have used the flash-lag effect with accompanying sounds (Vroomen & de Gelder, 2004b), visual temporal order judgment (TOJ) tasks with accompanying sounds Morein-Zamir et al, 2003;Vroomen & Keetels, 2006), sensorimotor synchronization Repp, 2005;Repp & Penel, 2002;Stekelenburg, Sugano & Vroomen, 2011;Sugano, Keetels & Vroomen, 2010, 2012, and other variants of cross-modal temporal capture (Alais & Burr, 2004b;Bruns & Getzmann, 2008;Chen & Zhou, 2011;Freeman & Driver, 2008;Getzmann, 2007;Kafaligonul & Stoner, 2010;Vroomen & de Gelder, 2000, 2003.…”

Spatial References and Audio-Tactile Interaction in Cross-Modal Dynamic Capture

“…'Cross-modal dynamic capture' (CDC) has been widely investigated for more than a decade (Chen and Zhou, 2011;Jiang and Chen, 2013;Sanabria et al, 2005Sanabria et al, , 2007aSoto-Faraco et al, 2002. In a typical 'capture' task, participants judged the direction of apparent motion in the target modality (such as auditory motion), ignoring the stimuli in the distractor modality (such as tactile motion).…”