The Motor Network Reduces Multisensory Illusory Perception

Abstract

Observing mouth movements has strikingly effects on the perception of speech. Any mismatch between sound and mouth movements will result in listeners perceiving illusory consonants (McGurk effect), whereas matching mouth movements assist with the correct recognition of speech sounds. Recent neuroimaging studies have yielded evidence that the motor areas are involved in speech processing, yet their contributions to multisensory illusion remain unclear. Using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) in an event-related design, we aimed to identify the functional roles of the motor network in the occurrence of multisensory illusion in female and male brains. fMRI showed bilateral activation of the inferior frontal gyrus (IFG) in audiovisually incongruent trials. Activity in the left IFG was negatively correlated with occurrence of the McGurk effect. The effective connectivity between the left IFG and the bilateral precentral gyri was stronger in incongruent than in congruent trials. The McGurk effect was reduced in incongruent trials by applying single-pulse TMS to motor cortex (M1) lip areas, indicating that TMS facilitates the left IFG-precentral motor network to reduce the McGurk effect. TMS of the M1 lip areas was effective in reducing the McGurk effect within the specific temporal range from 100 ms before to 200 ms after the auditory onset, and TMS of the M1 foot area did not influence the McGurk effect, suggesting topographical specificity. These results provide direct evidence that the motor network makes specific temporal and topographical contributions to the processing of multisensory integration of speech to avoid illusion.

Audio-visual combination of syllables involves time-sensitive dynamics following from fusion failure

“…Several previous studies have tied the IFG to AV fusion but its specific contribution is still unclear 15,26,59,60 . The current results suggest that the early IFG involvement in AV integration does not relate to feature identification 61 , but are specific to AV timing. We found that the IFG activity tracked temporal AV asynchrony, at least within the range used in the experiment (from -120 ms auditory lead to 320 ms auditory lag).…”

Cortical tracking of speech reveals top-down reconstructive processes

“…Indeed, unimodal neural entrainment to speech envelope or mouth movements is driven by neural sources located in the left motor and premotor cortex (Park et al, 2015;Park et al, 2018). A precentral origin of these top-down modulations is also suggested by the fact that corticobulbar excitability is modulated by passive listening to speech (Fadiga et al, 2002;Watkins et al, 2003;D'Ausilio et al, 2014;Schmitz et al, 2018) and that transient perturbation of the activity in premotor and motor areas produces somatotopically organized modulation of speech discrimination performance (D'Ausilio et al, 2009;Mottonen et al, 2009;Sato et al, 2009;D'Ausilio et al, 2012;Bartoli et al, 2015;Murakami et al, 2018).…”

Reduced resting state functional connectivity with increasing age-related hearing loss and McGurk susceptibility

“…For correlation analyses of brain connectivity with McGurk illusion rates, the same auditory seed was used as well as two other areas related to audio-visual integration, the left motor lip area and the left STS. The seed coordinates for the lip area were based on the average individual activation peaks in response to the McGurk task from Murakami and associates 41 . For the seed in the left STS, coordinates from the same study, used in an effective connectivity measure, were adopted.…”

“…Early auditory and visual areas are involved in the generation of the fused McGurk percept in healthy participants 38 . Additionally, areas crucial for speech production, including the lip area of the primary motor cortex, have been found to be involved in the perception of audio-visual speech stimuli [39][40][41][42] . Thereby, bottom-up sensory input is believed to be integrated with top-down information from frontal and motor areas in the posterior superior temporal sulcus (STS) 38,43,44 .…”

“…Concerning the neuroimaging data, we expected variability in resting state connectivity of the auditory cortices 13,21 as well as the DAN 13,20 , DMN 13 , and salience network 13,19 to be explained by progressing hearing loss. Further, we assumed that resting state functional coupling of audio-visual integration related areas, such as the primary motor cortex, STS, and auditory cortices, changes as a function of McGurk susceptibility 28,41,45 .…”