Sensory and Striatal Areas Integrate Auditory and Visual Signals into Behavioral Benefits during Motion Discrimination

Saldern, Sebastian von; Noppeney, Uta

doi:10.1523/jneurosci.3020-12.2013

Cited by 35 publications

(36 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For typical participants (group size n=7) auditory direction could be decoded from the pattern of right LOC responses but not hMT+. However, another study that examined a similar question did successfully classify the direction of auditory motion from hMT+ activity (Dormal, Rezk, Yakobov, Lepore, & Collignon, 2016), adding to several other studies that documented audiovisual interactions in this region (von Saldern & Noppeney, 2013). The involvement of right LOC in coding for auditory-motion direction was also indicated by Alink et al (2012) via a whole-brain searchlight-based pattern classification.…”

Section: Sensitivity To Regularity In Multimodal Contextsmentioning

confidence: 67%

Visual cortex signals a mismatch between regularity of auditory and visual streams

2017

View full text Add to dashboard Cite

Understanding how humans code for and respond to environmental uncertainty/regularity is a question shared by current computational and neurobiological approaches to human cognition. To date, studies investigating neurobiological systems that track input uncertainty have examined responses to uni-sensory streams. It is not known, however, whether there exist brain systems that combine information about the regularity of input streams presented to different senses. We report an fMRI study that aimed to identify brain systems that relate statistical information across sensory modalities. We constructed temporally extended auditory and visual streams, each of which could be random or highly regular, and presented them concurrently. We found strong signatures of "regularity matching" in visual cortex bilaterally; responses were higher when the level of regularity in the auditory and visual streams mismatched than when it matched, [(AudHigh/VisLow and AudLow/VisHigh) > (AudLow/VisLow and AudHigh/VisHigh)]. In addition, several frontal and parietal regions tracked regularity of the auditory or visual stream independently of the other stream's regularity. An individual-differences analysis showed that signatures of single-modality-focused regularity tracking in these fronto-parietal regions are inversely related to signatures of regularity-matching in visual cortex. Our findings suggest that i) visual cortex is a junction for integration of temporally-extended auditory and visual inputs and that ii) multisensory regularity-matching depends on balanced processing of both input modalities. We discuss the implications of these findings for neurobiological models of uncertainty and for understanding computations that underlie multisensory interactions in occipital cortex.3

show abstract

Section: Sensitivity To Regularity In Multimodal Contextsmentioning

confidence: 67%

Visual cortex signals a mismatch between regularity of auditory and visual streams

2017

View full text Add to dashboard Cite

show abstract

“…There are various approaches that employ the additive model, particularly in fMRI analysis, that are not associated with these confounds (e.g. Werner & Noppeney, 2010;Saldern, & Noppeney, 2013).…”

Section: Discussionmentioning

confidence: 99%

The effects of stereo disparity on the behavioural and electrophysiological correlates of perception of audio–visual motion in depth

et al. 2015

View full text Add to dashboard Cite

Motion is represented by low-level signals, such as size-expansion in vision or loudness changes in the auditory modality. The visual and auditory signals from the same object or event may be integrated and facilitate detection. We explored behavioural and electrophysiological correlates of congruent and incongruent audio-visual depth motion in conditions where auditory level changes, visual expansion, and visual disparity cues were manipulated. In Experiment 1 participants discriminated auditory motion direction whilst viewing looming or receding, 2D or 3D, visual stimuli. Responses were faster and more accurate for congruent than for incongruent audio-visual cues, and the congruency effect (i.e., difference between incongruent and congruent conditions) was larger for visual 3D cues compared to 2D cues. In Experiment 2, event-related potentials (ERPs) were collected during presentation of the 2D and 3D, looming and receding, audio-visual stimuli, while participants Page 2 of 44 detected an infrequent deviant sound. Our main finding was that audio-visual congruity was affected by retinal disparity at an early processing stage (135 -160 ms) over occipito-parietal scalp. Topographic analyses suggested that similar brain networks were activated for the 2D and 3D congruity effects, but that cortical responses were stronger in the 3D condition.Differences between congruent and incongruent conditions were observed between 140 -200 ms, 220 -280 ms, and 350 -500 ms after stimulus onset.

show abstract

“…Multisensory enhancement, which is the most reliable index of multisensory integration (and will be discussed here most extensively) may reflect computations that yield response magnitudes that are equal to, less than or greater than the sum of the responses to the individual component stimuli 7 . In behaviour, performance enhancements are often quantified by evaluating differences in the accuracy and speed of detection, localization and/or identification of stimuli 3,4,10–26 . In short, multisensory integration refers to a broad class of computations involving multiple sensory modalities in which information is integrated to produce an enhanced (or degraded) response.…”

Section: Defining Multisensory Integrationmentioning

confidence: 99%

Development of multisensory integration from the perspective of the individual neuron

2014

View full text Add to dashboard Cite

The ability to use cues from multiple senses in concert is a fundamental aspect of brain function. It maximizes the brain’s use of the information available to it at any given moment and enhances the physiological salience of external events. Because each sense conveys a unique perspective of the external world, synthesizing information across senses affords computational benefits that cannot otherwise be achieved. Multisensory integration not only has substantial survival value but can also create unique experiences that emerge when signals from different sensory channels are bound together. However, neurons in a newborn’s brain are not capable of multisensory integration, and studies in the midbrain have shown that the development of this process is not predetermined. Rather, its emergence and maturation critically depend on cross-modal experiences that alter the underlying neural circuit in such a way that optimizes multisensory integrative capabilities for the environment in which the animal will function.

show abstract

Sensory and Striatal Areas Integrate Auditory and Visual Signals into Behavioral Benefits during Motion Discrimination

Cited by 35 publications

References 51 publications

Visual cortex signals a mismatch between regularity of auditory and visual streams

Visual cortex signals a mismatch between regularity of auditory and visual streams

The effects of stereo disparity on the behavioural and electrophysiological correlates of perception of audio–visual motion in depth

Development of multisensory integration from the perspective of the individual neuron

Contact Info

Product

Resources

About