Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions

Holle, Henning; Obleser, Jonas; Rueschemeyer, Shirley‐Ann; Gunter, Thomas C.

doi:10.1016/j.neuroimage.2009.08.058

Cited by 143 publications

(157 citation statements)

References 45 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Thus, these direct connections might underlie inhibitory control during perception of eye motion. Furthermore, these direct connections with left IFG nicely dovetail with fMRI findings demonstrating coactivations (Holle et al, 2010) and effective connectivity (Willems et al, 2009) of left pSTS and IFG during integration of speech and gestures. Finally, both left and right pSTS were connected with posterior parts of the ipsilateral IPL.…”

Section: Anatomical Fiber Connections Of Pstssupporting

confidence: 76%

“…No modulation by the social meaning of gaze shifts was found in left pSTS (or pMTG on either side). Accordingly, recent studies suggested that the left pSTS may process the causality of biological motion (Morris et al, 2008) and integrates biological motion with language (Holle et al, 2010;Willems et al, 2009), whereas the right pSTS is more specifically engaged by emotional and social information (Blakemore et al, 2004).…”

Section: Location and Functional Properties Of Pstsmentioning

confidence: 99%

See 1 more Smart Citation

Processing social aspects of human gaze: A combined fMRI-DTI study

2011

View full text Add to dashboard Cite

Human gaze is a critical social cue that can reveal intentions and dispositions of others. The right posterior superior temporal sulcus (pSTS) is thought to be critically involved in processing eye gaze information. We combined diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI) to identify direct neural connections of right pSTS and to determine how these connections are modulated by the social significance of perceived gaze shifts. Participants saw faces with direct or averted gaze during event-related fMRI. Half of these faces remained static, and half displayed a dynamic gaze shift either towards or away from the subject. Social attention (dynamic gaze shifts towards the observer) not only increased activity in right pSTS, but also its functional connectivity with the right anterior insula (aIns) and right fusiform gyrus (FG). However, direct fiber connections from pSTS were demonstrated by DTI for the right aIns, but not the right FG. Moreover, the right FG responded to eye motion irrespective of direction and social significance; whereas the right aIns was selectively sensitive to social significance (i.e. gaze shifts towards the observer), but not generally to eye motion. We conclude that the social aspects of mutual gaze contact are processed by direct fiber pathways between right pSTS and right aIns; whereas increased connectivity with FG could reflect an enhanced perceptual analysis of changing facial features in dynamic gaze conditions and involves indirect fiber pathways with pSTS, perhaps via motion-selective regions in middle temporal (MT) gyrus that exhibited strong white-matter connections with both pSTS and FG and could thus provide inputs to these two areas.

show abstract

Section: Anatomical Fiber Connections Of Pstssupporting

confidence: 76%

Section: Location and Functional Properties Of Pstsmentioning

confidence: 99%

Processing social aspects of human gaze: A combined fMRI-DTI study

2011

View full text Add to dashboard Cite

show abstract

“…The language network, together with inferior parietal cortex, is also engaged during sign language production (Emmorey et al, 2007;MacSweeney et al, 2008). Importantly, the first whole-brain pattern shows fronto-temporal areas that have repeatedly been found in studies of co-speech gesture perception, such as left inferior frontal gyrus and left superior temporal gyrus (Dick et al, 2012;Green et al, 2009;Holle et al, 2008Holle et al, , 2010Willems et al, 2007Willems et al, , 2009). In addition, these areas are associated with semantic processes, such as word retrieval (left inferior frontal gyrus: Amunts et al, 2004;de Zubicaray & McMahon, 2009;Thompson-Schill et al, 1999;Tremblay & Gracco, 2006; superior temporal gyrus: Graves et al, 2010;Heath et al, 2012;Khader et al, 2010).…”

Section: Discussionmentioning

confidence: 79%

“…Their coordination with speech is both temporal-kinetic and semantic (Kita & Özyürek, 2003;Loehr, 2007) and they have been shown to affect learning and memory in the listener as well as the speaker (Goldin-Meadow, 2003;Hostetter, 2011;Marstaller & Burianová, 2013). Previous neuroimaging studies have found that the observation of co-speech gestures engages superior and middle temporal gyrus, intraparietal sulcus, and inferior frontal gyrus (Dick et al, 2009;Holle et al, 2008Holle et al, , 2010Hubbard et al, 2009;Kircher et al, 2009;Skipper et al, 2007Skipper et al, , 2009Straube et al, 2011;Willems et al, 2007Willems et al, , 2009). The findings from these studies strongly suggest that during the observation of co-speech gestures, frontal and temporal regions are engaged in semantic processing, whereas frontal and parietal areas are activated for action understanding (Marstaller & Burianová, 2014).…”

Section: Introductionmentioning

confidence: 99%

A common functional neural network for overt production of speech and gesture

Marstaller¹,

Burianová²

2015

Neuroscience

View full text Add to dashboard Cite

The perception of co-speech gestures, i.e., hand movements that co-occur with speech, has been investigated by several studies. The results show that the perception of co-speech gestures engages a core set of frontal, temporal, and parietal areas.However, no study has yet investigated the neural processes underlying the production of co-speech gestures. Specifically, it remains an open question whether Broca's area is central to the coordination of speech and gestures as has been suggested previously. The objective of this study was to use fMRI to (i) investigate the regional activations underlying overt production of speech, gestures, and cospeech gestures, and (ii) examine functional connectivity with Broca's area. We hypothesized that co-speech gesture production would activate frontal, temporal, and parietal regions that are similar to areas previously found during co-speech gesture perception and that both speech and gesture as well as co-speech gesture production would engage a neural network connected to Broca's area. Whole-brain analysis confirmed our hypothesis and showed that co-speech gesturing did engage brain areas that form part of networks known to subserve language and gesture. Functional connectivity analysis further revealed a functional network connected to Broca's area that is common to speech, gesture, and co-speech gesture production. This network consists of brain areas that play essential roles in motor control, suggesting that the coordination of speech and gesture is mediated by a shared motor control network.Our findings thus lend support to the idea that speech can influence co-speech gesture production on a motoric level.

show abstract

“…Moreover, it is interesting to consider that the perisylvian region is also involved in the integration of iconic gestures and speech (Holle, Obleser, Rueschemeyer, & Gunter, 2010;Straube, Green, Bromberger, & Kircher, 2011; for reviews see also Andric & Small, 2012;Marstaller & Burianov a, 2014). Furthermore, the posterior STS has been reported to be part of the action observation network (Georgescu et al, 2014) and incorporated in a neural network activated in social interaction (Leube et al, 2012).…”

Section: Visual Exploration Behaviour In Aphasic Patientsmentioning

confidence: 99%