Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion

Saint-Amour, Dave; Sanctis, Pierfilippo De; Molholm, Sophie; Ritter, Walter; Foxe, John J.

doi:10.1016/j.neuropsychologia.2006.03.036

Cited by 134 publications

(140 citation statements)

References 60 publications

Supporting

Mentioning

130

Contrasting

Order By: Relevance

“…This system seems to be dissociated from auditory spatial processing (27). Nevertheless, our results indicate significant functional connectivity with the transverse temporal gyrus, most likely supporting a bottom-up process underpinning a multisensory component (28). In addition, coactivation, and thus connectivity, in these areas has been correlated with working memory tasks (29).…”

Section: Discussionmentioning

confidence: 57%

Musical expertise is related to altered functional connectivity during audiovisual integration

Paraskevopoulos

Kraneburg

Herholz

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The present study investigated the cortical large-scale functional network underpinning audiovisual integration via magnetoencephalographic recordings. The reorganization of this network related to long-term musical training was investigated by comparing musicians to nonmusicians. Connectivity was calculated on the basis of the estimated mutual information of the sources' activity, and the corresponding networks were statistically compared. Nonmusicians' results indicated that the cortical network associated with audiovisual integration supports visuospatial processing and attentional shifting, whereas a sparser network, related to spatial awareness supports the identification of audiovisual incongruences. In contrast, musicians' results showed enhanced connectivity in regions related to the identification of auditory pattern violations. Hence, nonmusicians rely on the processing of visual clues for the integration of audiovisual information, whereas musicians rely mostly on the corresponding auditory information. The large-scale cortical network underpinning multisensory integration is reorganized due to expertise in a cognitive domain that largely involves audiovisual integration, indicating long-term training-related neuroplasticity.functional connectivity | MEG | multisensory integration | cortical plasticity | musical training M ultisensory integration is of such importance for our understanding of the surrounding world that its cortical correlates interact with most of the neocortical regions, including the ones traditionally considered as unisensory (1). The cortical areas that are usually referred to as multisensory include the superior temporal sulcus, the intraparietal sulcus, and the frontal cortex (2). Nevertheless, recent evidence suggests that multisensory perception engages more widespread areas than what the classic modular approaches have so far assumed (1). Moreover, audiovisual integration emerges from a dynamic interplay of distributed regions operating in large-scale networks (3).The amount of shared information within these large-scale networks of distributed neuronal groups conceptualizes the notion of functional connectivity (4). Recent methodological advances allow the identification of networks that emerge from whole-brain analyses of neuroimaging data such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) (5) and provide fertile ground for studying the functional connectivity patterns of higher cognitive processes. These kinds of networks are modeled as graphs composed of nodes, which represent the cortical areas contributing to the network, and by edges, which represent the connections between the nodes. Each network has specific attributes that quantify connectivity organization (6). These graphs depict the functional connectome of the respective cognitive processes. Recent evidence suggests that functional connectivity networks may be reorganized by factors mediating neuroplasticity such as learning and development (7), altering information processing ...

show abstract

Section: Discussionmentioning

confidence: 57%

Musical expertise is related to altered functional connectivity during audiovisual integration

Paraskevopoulos

Kraneburg

Herholz

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…ERP studies have revealed that this audiovisual integration occurs during the early stages of perception. The mismatch negativity induced by the illusory phoneme change created by a McGurk effect is not delayed relative to a mismatch negativity recorded after a real auditory change (around 150-250 msec) and the topography of the MMR is similar in both cases (Saint-Amour, De Sanctis, Molholm, Ritter, & Foxe, 2007;Colin et al, 2002). Early cross-modal integration is confirmed by the difference recorded between ERPs to audiovisual speech and the sum of the ERPs to unimodal auditory and visual stimuli on the N1/P2 auditory complex (van Wassenhove, Grant, & Poeppel, 2005).…”

Section: Introductionmentioning

confidence: 73%

Hearing Faces: How the Infant Brain Matches the Face It Sees with the Speech It Hears

Bristow

Dehaene‐Lambertz

Mattout

et al. 2008

Journal of Cognitive Neuroscience

136

116

View full text Add to dashboard Cite

Abstract& Speech is not a purely auditory signal. From around 2 months of age, infants are able to correctly match the vowel they hear with the appropriate articulating face. However, there is no behavioral evidence of integrated audiovisual perception until 4 months of age, at the earliest, when an illusory percept can be created by the fusion of the auditory stimulus and of the facial cues (McGurk effect). To understand how infants initially match the articulatory movements they see with the sounds they hear, we recorded high-density ERPs in response to auditory vowels that followed a congruent or incongruent silently articulating face in 10-week-old infants. In a first experiment, we determined that auditory-visual integration occurs during the early stages of perception as in adults. The mismatch response was similar in timing and in topography whether the preceding vowels were presented visually or aurally. In the second experiment, we studied audiovisual integration in the linguistic (vowel perception) and nonlinguistic (gender perception) domain. We observed a mismatch response for both types of change at similar latencies. Their topographies were significantly different demonstrating that cross-modal integration of these features is computed in parallel by two different networks. Indeed, brain source modeling revealed that phoneme and gender computations were lateralized toward the left and toward the right hemisphere, respectively, suggesting that each hemisphere possesses an early processing bias. We also observed repetition suppression in temporal regions and repetition enhancement in frontal regions. These results underscore how complex and structured is the human cortical organization which sustains communication from the first weeks of life on. &

show abstract

“…Here there were extensive structured ERPs and relatively early latency activation to the visual speech stimuli (see also, Saint-Amour et al, 2007). In the present study, a large number of EEG sweeps was collected, increasing the signalto-noise ratios and affording good estimates of the ERPs.…”

Section: Limitations Of the Cdrsmentioning

confidence: 84%

Spatiotemporal dynamics of audiovisual speech processing

Bernstein

Auer

Wagner³

et al. 2008

NeuroImage

View full text Add to dashboard Cite

The cortical processing of auditory-alone, visual-alone, and audiovisual speech information is temporally and spatially distributed, and functional magnetic resonance imaging (fMRI) cannot adequately resolve its temporal dynamics. In order to investigate a hypothesized spatio-temporal organization for audiovisual speech processing circuits, event-related potentials (ERPs) were recorded using electroencephalography (EEG). Stimuli were congruent audiovisual /bα/, incongruent auditory /bα/ synchronized with visual /gα/, auditory-only /bα/, and visual-only /bα/ and /gα/. Current density reconstructions (CDRs) of the ERP data were computed across the latency interval of 50-250 milliseconds. The CDRs demonstrated complex spatio-temporal activation patterns that differed across stimulus conditions. The hypothesized circuit that was investigated here comprised initial integration of audiovisual speech by the middle superior temporal sulcus (STS), followed by recruitment of the intraparietal sulcus (IPS), followed by activation of Broca's area (Miller and d'Esposito, 2005). The importance of spatio-temporally sensitive measures in evaluating processing pathways was demonstrated. Results showed, strikingly, early (< 100 msec) and simultaneous activations in areas of the supramarginal and angular gyrus (SMG/AG), the IPS, the inferior frontal gyrus, and the dorsolateral prefrontal cortex. Also, emergent left hemisphere SMG/AG activation, not predicted based on the unisensory stimulus conditions was observed at approximately 160 to 220 msec. The STS was neither the earliest nor most prominent activation site, although it is frequently considered the sine qua non of audiovisual speech integration. As discussed here, the relatively late activity of the SMG/AG solely under audiovisual conditions is a possible candidate audiovisual speech integration response.

show abstract

Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion

Cited by 134 publications

References 60 publications

Musical expertise is related to altered functional connectivity during audiovisual integration

Musical expertise is related to altered functional connectivity during audiovisual integration

Hearing Faces: How the Infant Brain Matches the Face It Sees with the Speech It Hears

Spatiotemporal dynamics of audiovisual speech processing

Contact Info

Product

Resources

About