Distinct roles of temporal and frontoparietal cortex in representing actions across vision and language

Wurm, Moritz F.; Caramazza, Alfonso

doi:10.1038/s41467-018-08084-y

Cited by 91 publications

(102 citation statements)

References 65 publications

(76 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once again, this research has highlighted activity in a brain network encompassing the MTG. The involvement in processing both words and gestures is consistent with the proposal that the entire MTG functions as a multimodal interface, interposed between the inferior temporal and the superior temporal gyri, specialized to processing of pictorial and verbal material, respectively (Binder et al, 2009;Visser et al, 2012;Wurm and Caramazza, 2019). While previous studies suggest that the MTG is implicated in representing the meaning of communicative signs, it is unclear how neural information is organized; in particular, whether neural representation of meanings in the MTG is abstracted away from the input modality, or rather reflects, in some way, whether meanings are conveyed through words or gestures.…”

Section: Introductionsupporting

confidence: 85%

“…We emphasize that modality effects in the MTG reflect fine-grained distinctions underneath the general meaningful versus meaningless distinction. Primarily, the MTG responds more strongly to meaningful compared with meaningless stimuli; as such, it is a multimodal area (Binder et al, 2009;Visser et al, 2012;Wurm and Caramazza, 2019).…”

Section: Meaningful Stimuli: Pantomimes Emblems and Wordsmentioning

confidence: 99%

See 1 more Smart Citation

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

2019

View full text Add to dashboard Cite

The middle temporal gyrus (MTG) has been shown to be recruited during the processing of words, but also during the observation of actions. Here we investigated how information related to words and gestures is organized along the MTG. To this aim, we measured the BOLD response in the MTG to video clips of gestures and spoken words in 17 healthy human adults (male and female). Gestures consisted of videos of an actress performing object-use pantomimes (iconic representations of object-directed actions; e.g., playing guitar), emblems (conventional gestures, e.g., thumb up), and meaningless gestures. Word stimuli (verbs, nouns) consisted of video clips of the same actress pronouncing words. We found a stronger response to meaningful compared with meaningless gestures along the whole left and large portions of the right MTG. Importantly, we observed a gradient, with posterior regions responding more strongly to gestures (pantomimes and emblems) than words and anterior regions showing a stronger response to words than gestures. In an intermediate region in the left hemisphere, the response was significantly higher to words and emblems (i.e., items with a greater arbitrariness of the sign-to-meaning mapping) than to pantomimes. These results show that the large-scale organization of information in the MTG is driven by the input modality and may also reflect the arbitrariness of the relationship between sign and meaning.Here we investigated the organizing principle of information in the middle temporal gyrus, taking into consideration the inputmodality and the arbitrariness of the relationship between a sign and its meaning. We compared the middle temporal gyrus response during the processing of pantomimes, emblems, and spoken words. We found that posterior regions responded more strongly to pantomimes and emblems than to words, whereas anterior regions responded more strongly to words than to pantomimes and emblems. In an intermediate region, only in the left hemisphere, words and emblems evoked a stronger response than pantomimes. Our results identify two organizing principles of neural representation: the modality of communication (gestural or verbal) and the (arbitrariness of the) relationship between sign and meanings.

show abstract

Section: Introductionsupporting

confidence: 85%

Section: Meaningful Stimuli: Pantomimes Emblems and Wordsmentioning

confidence: 99%

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

2019

View full text Add to dashboard Cite

show abstract

“…Functional magnetic resonance imaging (fMRI) findings in neurotypical adults are consistent with the available neuropsychological data, as there is increased blood oxygen level-dependent (BOLD) contrast in the left posterior middle temporal gyrus when participants gesture tool use (Brandi et al, 2014;Johnson-Frey et al, 2005;Vry et al, 2015), view images of tools (Beauchamp et al, 2002;Chao et al, 1999;Garcea et al, 2016;Mahon et al, 2007; for review, see Lingnau & Downing, 2015;A. Martin, 2007), and make judgments about actions (Kable et al, 2005;Kable et al, 2002;Wurm & Caramazza, 2019), including tool use actions (Kleineberg et al, 2018;Valyear & Culham, 2010). Furthermore, an emerging literature demonstrates there is increased functional connectivity between the left inferior parietal lobule and left posterior middle temporal gyrus when neurotypical participants gesture the use of tools (Garcea et al, 2018;Hutchison & Gallivan, 2018;Vingerhoets & Clauwaert, 2015).…”

Section: Introductionmentioning

confidence: 56%

Structural Disconnection of the Tool Use Network After Left Hemisphere Stroke Predicts Limb Apraxia Severity

Garcea

Greene

Grafton

et al. 2020

Preprint

View full text Add to dashboard Cite

Producing a tool use gesture is a complex process drawing upon the integration of stored knowledge of tools and their associated actions with sensory-motor mechanisms supporting the planning and control of hand and arm actions. Understanding how sensory-motor systems in parietal cortex interface with semantic representations of actions and objects in the temporal lobe remains a critical issue, and is hypothesized to be a key determinant of the severity of limb apraxia, a deficit in producing skilled action after left hemisphere stroke. We used voxel-based and connectome-based lesion symptom mapping with data from 57 left hemisphere stroke participants to assess the lesion sites and structural disconnection patterns associated with poor tool use gesturing. We found that structural disconnection between the left inferior parietal lobule, lateral temporal lobe (left middle temporal gyrus) and ventral temporal cortex (left medial fusiform gyrus) predicted the severity of tool use gesturing performance. Control analyses demonstrated that reductions in right-hand grip strength were associated with motor system disconnection, bypassing regions supporting tool use gesturing. Our findings provide causal evidence that limb apraxia may arise, in part, from disconnection of conceptual representations in the temporal lobe from mechanisms enabling skilled action production in the inferior parietal lobule.

show abstract

“…that can be used by an agent in order to reason about the possibility to act within and upon a context, in a proper and agent's consistent-with-intention way 27,28 . The action reappraisal idea appears to be supported by recent neuropsychological evidence indicating the inferior parietal cortex and the middle temporal brain areas as regions where a multimodal integration of action and semantic information takes place to generate high-level cognitive representations about tools 35,[37][38][39] .…”

mentioning

confidence: 91%

Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Federico

Brandimonte

2020

Sci Rep

View full text Add to dashboard Cite

Alongside language and bipedal locomotion, tool use is a characterizing activity of human beings. Current theories in the field embrace two contrasting approaches: "manipulation-based" theories, which are anchored in the embodied-cognition view, explain tool use as deriving from past sensorimotor experiences, whereas "reasoning-based" theories suggest that people reason about object properties to solve everyday-life problems. Here, we present results from two eye-tracking experiments in which we manipulated the visuo-perceptual context (thematically consistent vs. inconsistent object-tool pairs) and the goal of the task (free observation or looking to recognise). We found that participants exhibited reversed tools' visual-exploration patterns, focusing on the tool's manipulation area under thematically consistent conditions and on its functional area under thematically inconsistent conditions. Crucially, looking at the tools with the aim of recognising them produced longer fixations on the tools' functional areas irrespective of thematic consistency. In addition, tools (but not objects) were recognised faster in the thematically consistent conditions. These results strongly support reasoning-based theories of tool use, as they indicate that people primarily process semantic rather than sensorimotor information to interact with the environment in an agent's consistent-with-goal way. Such a pre-eminence of semantic processing challenges the mainstream embodied-cognition view of human tool use.

show abstract

Distinct roles of temporal and frontoparietal cortex in representing actions across vision and language

Cited by 91 publications

References 65 publications

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

Structural Disconnection of the Tool Use Network After Left Hemisphere Stroke Predicts Limb Apraxia Severity

Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Contact Info

Product

Resources

About