Crossmodal interactions and multisensory integration in the perception of audio-visual motion — A free-field study

Schmiedchen, Kristina; Freigang, Claudia; Nitsche, Ines; Rübsamen, Rudolf

doi:10.1016/j.brainres.2012.05.015

Cited by 25 publications

(23 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental protocol (including the recording of hearing thresholds, the participants' task, and the pointing procedure) was adapted from our previous localization study (Schmiedchen et al, 2012). Participants were tested in complete darkness.…”

Section: Methodsmentioning

confidence: 99%

“…The auditory and visual stimuli were digitally generated using RPvdsEx (real-time processor visual design studio, Tucker Davis Technologies [TDT]) and delivered to two multichannel signal processors (RX8, System3, TDT). A more detailed description of the experimental setup and the calibration of the loudspeakers is given in Schmiedchen, Freigang, Nitsche, and Rübsamen (2012).…”

Section: Apparatus and Stimulimentioning

confidence: 99%

See 1 more Smart Citation

A comparison of visual and auditory representational momentum in spatial tasks

Schmiedchen

Freigang

Rübsamen

et al. 2013

Atten Percept Psychophys

Self Cite

View full text Add to dashboard Cite

Similarities have been observed in the localization of the final position of moving visual and moving auditory stimuli: Perceived endpoints that are judged to be farther in the direction of motion in both modalities likely reflect extrapolation of the trajectory, mediated by predictive mechanisms at higher cognitive levels. However, actual comparisons of the magnitudes of displacement between visual tasks and auditory tasks using the same experimental setup are rare. As such, the purpose of the present free-field study was to investigate the influences of the spatial location of motion offset, stimulus velocity, and motion direction on the localization of the final positions of moving auditory stimuli (Experiment 1 and 2) and moving visual stimuli (Experiment 3). To assess whether auditory performance is affected by dynamically changing binaural cues that are used for the localization of moving auditory stimuli (interaural time differences for low-frequency sounds and interaural intensity differences for high-frequency sounds), two distinct noise bands were employed in Experiments 1 and 2. In all three experiments, less precise encoding of spatial coordinates in paralateral space resulted in larger forward displacements, but this effect was drowned out by the underestimation of target eccentricity in the extreme periphery. Furthermore, our results revealed clear differences between visual and auditory tasks. Displacements in the visual task were dependent on velocity and the spatial location of the final position, but an additional influence of motion direction was observed in the auditory tasks. Together, these findings indicate that the modalityspecific processing of motion parameters affects the extrapolation of the trajectory.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Apparatus and Stimulimentioning

confidence: 99%

A comparison of visual and auditory representational momentum in spatial tasks

Schmiedchen

Freigang

Rübsamen

et al. 2013

Atten Percept Psychophys

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is mostly achieved by using a pointing task, where participants are instructed to indicate the perceived position of a sound source (Shankweiler 1961;Seeber et al 2010;Kerber and Seeber 2012;Kühnle et al 2012;Schmiedchen et al 2012Schmiedchen et al , 2013Freigang et al 2014a) or by alignment of the participant's head and gaze to the direction of the sound source (Lewald et al 2000). Another paradigm used to measure the localisation accuracy is the absolute identification task (Abel et al 2000), where participants have to indicate the sound direction by choosing one of several indicated sound source positions in an nalternative forced-choice task.…”

Section: Measuring Auditory Space Processingmentioning

confidence: 99%

Age-related changes in sound localisation ability

et al. 2015

Self Cite

View full text Add to dashboard Cite

Auditory spatial processing is an important ability in everyday life and allows the processing of omnidirectional information. In this review, we report and compare data from psychoacoustic and electrophysiological experiments on sound localisation accuracy and auditory spatial discrimination in infants, children, and young and older adults. The ability to process auditory spatial information changes over lifetime: the perception of the acoustic space develops from an initially imprecise representation in infants and young children to a concise representation of spatial positions in young adults and the respective performance declines again in older adults. Localisation accuracy shows a strong deterioration in older adults, presumably due to declined processing of binaural temporal and monaural spectro-temporal cues. When compared to young adults, the thresholds for spatial discrimination were strongly elevated both in young children and older adults. Despite the consistency of the measured values the underlying causes for the impaired performance might be different: (1) the effect is due to reduced cognitive processing ability and is thus task-related; (2) the effect is due to reduced information about the auditory space and caused by declined processing in auditory brain stem circuits; and (3) the auditory space processing regime in young children is still undergoing developmental changes and the interrelation with spatial visual processing is not yet established. In conclusion, we argue that for studying auditory space processing over the life course, it is beneficial to investigate spatial discrimination ability instead of localisation accuracy because it more reliably indicates changes in the processing ability.

show abstract

“…While there is an emerging consensus that the underlying neural correlates likely involve multiple stages of the sensory decision making pathways, it remains a challenge to uncover the dynamic processes that implement the multisensory benefit for an upcoming decision in the human brain (Bizley et al, 2016, Kayser and Shams, 2015, Rohe and Noppeney, 2014, Rohe and Noppeney, 2016). For example, many studies have shown that judgements about visual motion can be influenced by simultaneous sounds (Alais and Burr, 2004, Beer and Roder, 2004, Lewis and Noppeney, 2010, Schmiedchen et al, 2012) or vestibular information (Fetsch et al, 2010, Gu et al, 2008), even so when the multisensory stimulus is not directly task relevant (Gleiss and Kayser, 2014b, Kim et al, 2012, Sekuler et al, 1997). In particular, congruent multisensory evidence enhances visual motion discrimination performance over incongruent multisensory information (Meyer and Wuerger, 2001, Meyer et al, 2005, Soto-Faraco et al, 2003, Soto-Faraco et al, 2002).…”

Section: Introductionmentioning

confidence: 99%

Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations

2017

View full text Add to dashboard Cite

Sensory discriminations, such as judgements about visual motion, often benefit from multisensory evidence. Despite many reports of enhanced brain activity during multisensory conditions, it remains unclear which dynamic processes implement the multisensory benefit for an upcoming decision in the human brain. Specifically, it remains difficult to attribute perceptual benefits to specific processes, such as early sensory encoding, the transformation of sensory representations into a motor response, or to more unspecific processes such as attention. We combined an audio-visual motion discrimination task with the single-trial mapping of dynamic sensory representations in EEG activity to localize when and where multisensory congruency facilitates perceptual accuracy. Our results show that a congruent sound facilitates the encoding of motion direction in occipital sensory - as opposed to parieto-frontal - cortices, and facilitates later - as opposed to early (i.e. below 100 ms) - sensory activations. This multisensory enhancement was visible as an earlier rise of motion-sensitive activity in middle-occipital regions about 350 ms from stimulus onset, which reflected the better discriminability of motion direction from brain activity and correlated with the perceptual benefit provided by congruent multisensory information. This supports a hierarchical model of multisensory integration in which the enhancement of relevant sensory cortical representations is transformed into a more accurate choice.

show abstract

Crossmodal interactions and multisensory integration in the perception of audio-visual motion — A free-field study

Cited by 25 publications

References 62 publications

A comparison of visual and auditory representational momentum in spatial tasks

A comparison of visual and auditory representational momentum in spatial tasks

Age-related changes in sound localisation ability

Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations

Contact Info

Product

Resources

About