Listening to talking faces: motor cortical activation during speech perception

Skipper, Jeremy I.; Nusbaum, Howard C.; Small, Steven L.

doi:10.1016/j.neuroimage.2004.11.006

Cited by 291 publications

(309 citation statements)

References 94 publications

Supporting

Mentioning

275

Contrasting

Order By: Relevance

“…These motor and premotor activations are in accordance with previous studies on auditory, visual, and audiovisual speech perception showing a key role of motor regions in speech processing (e.g., d'Ausilio et al, 2009d'Ausilio et al, , 2011Sato et al, 2009Sato et al, , 2010Möttönen & Watkins, 2009;Meister et al, 2007;Skipper et al, 2005Skipper et al, , 2007Pekkola et al, 2006;Pulvermuller et al, 2006;Wilson & Iacoboni, 2006;Ojanen et al, 2005;Callan et al, 2003Callan et al, , 2004Watkins & Paus, 2004;Wilson et al, 2004;Calvert & Campbell, 2003;Watkins et al, 2003;Campbell et al, 2001;Calvert et al, 2000). It is worthwhile noting that, in this study, participants were only asked to attentively listen to and/or watch speech stimuli.…”

Section: Visuo-lingual and Visuo-facial Speech Stimuli Share A Commonsupporting

confidence: 91%

“…Neuroimaging studies demonstrate the existence of specific brain areas playing a key role in the audiovisual integration of speech. Notably, activity within unisensory visual and auditory regions (the visual motion-sensitive cortex, V5/MT, and the Heschl's gyrus) as well as within multisensory regions (the posterior parts of the left superior temporal gyrus/STS [pSTS/pSTG]) is modulated during audiovisual speech perception, when compared with auditory and visual unimodal conditions (Skipper et al, 2007;Skipper, Nusbaum, & Small, 2005;Callan et al, 2003Callan et al, , 2004Calvert et al, 2000). Because pSTS/pSTG displays supra-additive and subadditive responses during congruent and incongruent stimuli presentation, it has been proposed that both visual and auditory speech information are integrated in these high-level multisensory integrative regions and that modulations of neuronal responses within the sensory-specific cortices would then be due to feedback projections from this multisensory region.…”

Section: Motor Resonance Extends To Speech Actionmentioning

confidence: 99%

See 1 more Smart Citation

Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions

Treille

Vilain

Hueber

et al. 2017

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

Abstract■ Action recognition has been found to rely not only on sensory brain areas but also partly on the observer's motor system. However, whether distinct auditory and visual experiences of an action modulate sensorimotor activity remains largely unknown. In the present sparse sampling fMRI study, we determined to which extent sensory and motor representations interact during the perception of tongue and lip speech actions. Tongue and lip speech actions were selected because tongue movements of our interlocutor are accessible via their impact on speech acoustics but not visible because of its position inside the vocal tract, whereas lip movements are both "audible" and visible. Participants were presented with auditory, visual, and audiovisual speech actions, with the visual inputs related to either a sagittal view of the tongue movements or a facial view of the lip movements of a speaker, previously recorded by an ultrasound imaging system and a video camera. Although the neural networks involved in visual visuo-lingual and visuo-facial perception largely overlapped, stronger motor and somatosensory activations were observed during visuo-lingual perception. In contrast, stronger activity was found in auditory and visual cortices during visuo-facial perception. Complementing these findings, activity in the left premotor cortex and in visual brain areas was found to correlate with visual recognition scores observed for visuo-lingual and visuo-facial speech stimuli, respectively, whereas visual activity correlated with RTs for both stimuli. These results suggest that unimodal and multimodal processing of lip and tongue speech actions rely on common sensorimotor brain areas. They also suggest that visual processing of audible but not visible movements induces motor and visual mental simulation of the perceived actions to facilitate recognition and/or to learn the association between auditory and visual signals. ■

show abstract

Section: Visuo-lingual and Visuo-facial Speech Stimuli Share A Commonsupporting

confidence: 91%

Section: Motor Resonance Extends To Speech Actionmentioning

confidence: 99%

Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions

Treille

Vilain

Hueber

et al. 2017

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

show abstract

“…The posterior portion of the STS included in the conjunctions, the so-called STS-ms, is a focal point for cross-modal sensory associations (28) and may play a role in integration of linguistic and paralinguistic information encoded in auditory and visual modalities (29,30). For this reason, it is not surprising that the region responds to meaningful gestural-visual as well as vocal-auditory stimuli.…”

Section: Shared Activations For Perception Of Symbolic Gesture and Spmentioning

confidence: 99%

Symbolic gestures and spoken language are processed by a common neural system

Gannon

Emmorey

et al. 2009

Proc. Natl. Acad. Sci. U.S.A.

228

193

View full text Add to dashboard Cite

Symbolic gestures, such as pantomimes that signify actions (e.g., threading a needle) or emblems that facilitate social transactions (e.g., finger to lips indicating ''be quiet''), play an important role in human communication. They are autonomous, can fully take the place of words, and function as complete utterances in their own right. The relationship between these gestures and spoken language remains unclear. We used functional MRI to investigate whether these two forms of communication are processed by the same system in the human brain. Responses to symbolic gestures, to their spoken glosses (expressing the gestures' meaning in English), and to visually and acoustically matched control stimuli were compared in a randomized block design. General Linear Models (GLM) contrasts identified shared and unique activations and functional connectivity analyses delineated regional interactions associated with each condition. Results support a model in which bilateral modality-specific areas in superior and inferior temporal cortices extract salient features from vocalauditory and gestural-visual stimuli respectively. However, both classes of stimuli activate a common, left-lateralized network of inferior frontal and posterior temporal regions in which symbolic gestures and spoken words may be mapped onto common, corresponding conceptual representations. We suggest that these anterior and posterior perisylvian areas, identified since the mid-19th century as the core of the brain's language system, are not in fact committed to language processing, but may function as a modality-independent semiotic system that plays a broader role in human communication, linking meaning with symbols whether these are

show abstract

“…One should be cautious, though, in predicting that this is the only network involved in audio-visual speech binding. Among other things, we do not yet know the extent to which the hypothesized posterior audio-visual stream is responsible for cross-modal integration of vision and audition in the perception of speech prosody, or for the interplay of vision and hearing in the perception of spoken discourse (but see Skipper et al 2005).…”

Section: The Broader Picturementioning

confidence: 99%

The processing of audio-visual speech: empirical and neural bases

Campbell

2007

Phil. Trans. R. Soc. B

240

223

View full text Add to dashboard Cite

In this selective review, I outline a number of ways in which seeing the talker affects auditory perception of speech, including, but not confined to, the McGurk effect. To date, studies suggest that all linguistic levels are susceptible to visual influence, and that two main modes of processing can be described: a complementary mode, whereby vision provides information more efficiently than hearing for some under-specified parts of the speech stream, and a correlated mode, whereby vision partially duplicates information about dynamic articulatory patterning.Cortical correlates of seen speech suggest that at the neurological as well as the perceptual level, auditory processing of speech is affected by vision, so that 'auditory speech regions' are activated by seen speech. The processing of natural speech, whether it is heard, seen or heard and seen, activates the perisylvian language regions (leftOright). It is highly probable that activation occurs in a specific order. First, superior temporal, then inferior parietal and finally inferior frontal regions (leftOright) are activated. There is some differentiation of the visual input stream to the core perisylvian language system, suggesting that complementary seen speech information makes special use of the visual ventral processing stream, while for correlated visual speech, the dorsal processing stream, which is sensitive to visual movement, may be relatively more involved.

show abstract

Listening to talking faces: motor cortical activation during speech perception

Cited by 291 publications

References 94 publications

Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions

Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions

Symbolic gestures and spoken language are processed by a common neural system

The processing of audio-visual speech: empirical and neural bases

Contact Info

Product

Resources

About