Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study

Molholm, Sophie; Ritter, Walter; Murray, Micah M.; Javitt, Daniel C.; Schroeder, Charles E.; Foxe, John J.

doi:10.1016/s0926-6410(02)00066-6

Cited by 790 publications

(733 citation statements)

References 55 publications

Supporting

Mentioning

633

Contrasting

Unclassified

Order By: Relevance

“…They suggested that, depending on the relative timing and/or location of multisensory inputs, neural responses can sometimes exceed (or fall below) the sum of the responses for each unisensory input (Lakatos et al, 2007). Nonlinear analysis criteria have also been applied to EEG data in some multisensory studies that typically manipulated presence/absence of costimulation in a second modality (Giard and Peronnet, 1999;Foxe et al, 2000;Fort et al, 2002;Molholm et al, 2002Molholm et al, , 2004Murray et al, 2005) rather than a detailed relationship in temporal patterning as here. Similar nonadditive criteria have even been applied to fMRI data (Calvert et al, 2001).…”

Section: Comparison Of Our Two Multisensory Conditions With the Unisementioning

confidence: 99%

See 1 more Smart Citation

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Noesselt

Rieger²,

Schoenfeld³

et al. 2007

J. Neurosci.

253

228

View full text Add to dashboard Cite

The brain should integrate related but not unrelated information from different senses. Temporal patterning of inputs to different modalities may provide critical information about whether those inputs are related or not. We studied effects of temporal correspondence between auditory and visual streams on human brain activity with functional magnetic resonance imaging (fMRI). Streams of visual flashes with irregularly jittered, arrhythmic timing could appear on right or left, with or without a stream of auditory tones that coincided perfectly when present (highly unlikely by chance), were noncoincident with vision (different erratic, arrhythmic pattern with same temporal statistics), or an auditory stream appeared alone. fMRI revealed blood oxygenation level-dependent (BOLD) increases in multisensory superior temporal sulcus (mSTS), contralateral to a visual stream when coincident with an auditory stream, and BOLD decreases for noncoincidence relative to unisensory baselines. Contralateral primary visual cortex and auditory cortex were also affected by audiovisual temporal correspondence or noncorrespondence, as confirmed in individuals. Connectivity analyses indicated enhanced influence from mSTS on primary sensory areas, rather than vice versa, during audiovisual correspondence. Temporal correspondence between auditory and visual streams affects a network of both multisensory (mSTS) and sensory-specific areas in humans, including even primary visual and auditory cortex, with stronger responses for corresponding and thus related audiovisual inputs.

show abstract

Section: Comparison Of Our Two Multisensory Conditions With the Unisementioning

confidence: 99%

“…This has become an emerging theme in recent multisensory work, using different neural measures (cf. Giard and Peronnet, 1999;Macaluso et al, 2000;Molholm et al, 2002;Ghazanfar et al, 2005;Miller and D'Esposito, 2005;Watkins et al, 2006;Kayser et al, 2007;Lakatos et al, 2007).…”

Section: Effects Of Audiovisual Temporal Correspondence On Primary Sementioning

confidence: 99%

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Noesselt

Rieger²,

Schoenfeld³

et al. 2007

J. Neurosci.

253

228

View full text Add to dashboard Cite

show abstract

“…The notion that multisensory integration is restricted to higherorder areas has recently been challenged by human and animal studies that have revealed that crossmodal interactions can occur in unisensory areas at very low levels of cortical processing (Buchel et al, 1998;Calvert et al, 1999Calvert et al, , 2001Macaluso et al, 2000;Schroeder et al, 2001;Amedi et al, 2002;Ghazanfar et al, 2005;Kriegstein et al, 2005;Miller and D'Esposito, 2005;Watkins et al, 2006;Martuzzi et al, 2007;Kayser et al, 2007Kayser et al, , 2008Romei et al, 2007Romei et al, , 2008Wang et al, 2008) and more importantly at very short latencies (Giard and Peronnet, 1999;Foxe et al, 2000;Molholm et al, 2002;Murray et al, 2005;Senkowski et al, 2007;Sperdin et al, 2009). Such a fast timing of multisensory interactions rule out an origin in the multisensory areas mediated through backward projections, and instead favor direct heteromodal connections.…”

Section: Heteromodal Connections: Connections Between Different Sensomentioning

confidence: 99%

“…Considering that only the peripheral visual field representation of V1 receives significant projections from the auditory cortex, such congruency in the spatial features may serve to facilitate gaze orienting, and consequently the relocation of foveal vision to peripheral locations in the visual field. However, it should also be mentioned that facilitative effects have been observed in humans when stimuli were centrally presented in both auditory and visual modalities (Giard and Peronnet, 1999;Molholm et al, 2002;Martuzzi et al, 2007). Ethologically, a role in alertness for dangerous stimuli is highly probable, an interpretation that can also be attributed to the visuo-somatosensory projections: the specific link between the FST visual complex and the representation of the face in the somatosensory cortex could contribute to phenomena of avoidance of a ''dangerous" stimulus which may hit the body Graziano, 2003, 2004).…”

Section: Specificity Of Heteromodal Connections: Ethological Rolementioning

confidence: 99%

Multisensory anatomical pathways

2009

View full text Add to dashboard Cite

In order to interact with the multisensory world that surrounds us, we must integrate various sources of sensory information (vision, hearing, touch. . .). A fundamental question is thus how the brain integrates the separate elements of an object defined by several sensory components to form a unified percept. The superior colliculus was the main model for studying multisensory integration. At the cortical level, until recently, multisensory integration appeared to be a characteristic attributed to high-level association regions. First, we describe recently observed direct cortico-cortical connections between different sensory cortical areas in the non-human primate and discuss the potential role of these connections. Then, we show that the projections between different sensory and motor cortical areas and the thalamus enabled us to highlight the existence of thalamic nuclei that, by their connections, may represent an alternative pathway for information transfer between different sensory and/or motor cortical areas. The thalamus is in position to allow a faster transfer and even an integration of information across modalities. Finally, we discuss the role of these non-specific connections regarding behavioral evidence in the monkey and recent electrophysiological evidence in the primary cortical sensory areas.

show abstract

“…This analysis principle has been well documented in audio-visual studies [19][20][21] . In this case, the sham-sound condition and the puff alone condition are added, as a coupled sham -speech-sound allows us to account for low amplitude nonspecific auditory responses demonstrated in Figure 1.…”

Section: Assessment Of Response To Multisensory Protocol: Auditory-tamentioning

confidence: 82%

Quantitative Assessment of Cortical Auditory-tactile Processing in Children with Disabilities

Maitre

Key

2014

JoVE

View full text Add to dashboard Cite

Objective and easy measurement of sensory processing is extremely difficult in nonverbal or vulnerable pediatric patients. We developed a new methodology to quantitatively assess children's cortical processing of light touch, speech sounds and the multisensory processing of the 2 stimuli, without requiring active subject participation or causing children discomfort. To accomplish this we developed a dual channel, time and strength calibrated air puff stimulator that allows both tactile stimulation and sham control. We combined this with the use of event-related potential methodology to allow for high temporal resolution of signals from the primary and secondary somatosensory cortices as well as higher order processing. This methodology also allowed us to measure a multisensory response to auditory-tactile stimulation. Video LinkThe video component of this article can be found at

show abstract

Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study

Cited by 790 publications

References 55 publications

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Multisensory anatomical pathways

Quantitative Assessment of Cortical Auditory-tactile Processing in Children with Disabilities

Contact Info

Product

Resources

About