Symbolic gestures and spoken language are processed by a common neural system

Abstract

Symbolic gestures, such as pantomimes that signify actions (e.g., threading a needle) or emblems that facilitate social transactions (e.g., finger to lips indicating ''be quiet''), play an important role in human communication. They are autonomous, can fully take the place of words, and function as complete utterances in their own right. The relationship between these gestures and spoken language remains unclear. We used functional MRI to investigate whether these two forms of communication are processed by the same system in the human brain. Responses to symbolic gestures, to their spoken glosses (expressing the gestures' meaning in English), and to visually and acoustically matched control stimuli were compared in a randomized block design. General Linear Models (GLM) contrasts identified shared and unique activations and functional connectivity analyses delineated regional interactions associated with each condition. Results support a model in which bilateral modality-specific areas in superior and inferior temporal cortices extract salient features from vocalauditory and gestural-visual stimuli respectively. However, both classes of stimuli activate a common, left-lateralized network of inferior frontal and posterior temporal regions in which symbolic gestures and spoken words may be mapped onto common, corresponding conceptual representations. We suggest that these anterior and posterior perisylvian areas, identified since the mid-19th century as the core of the brain's language system, are not in fact committed to language processing, but may function as a modality-independent semiotic system that plays a broader role in human communication, linking meaning with symbols whether these are

Neural systems supporting linguistic structure, linguistic experience, and symbolic communication in sign language and gesture

“…This network is consistent with previous studies of both sign language and gesture that used similar contrasts with a low-level baseline (10,(22)(23)(24)(25)(26)(27). Critically, however, compared with the backwardlayered conditions that controlled for stimulus features (such as biological motion and face perception) and for motor responses, a much more restricted set of brain areas was activated, with considerably less overlap across groups and conditions.…”

“…In signers, activation for both sign language and pantomime gestures was reported in classical language-related areas including the IFG, the posterior region of the STS (STSp), and the SMG, although typically these activations are stronger for sign language than gesture. Similar patterns of activation-although often more bilateral-have been observed in nonsigners, for meaningful as well as for meaningless gesture perception (24)(25)(26)(27).…”

“…Previous studies had suggested that both gesture and sign language engage a broad, common network of brain regions including classical left hemisphere language areas. Some previous studies used pantomimed actions (10,21,27), which are more literal and less abstractly symbolic than some of the gestures used in the present study; other studies used gestures with established meanings [emblems (22,27,30)], or meaningless gestures (21,23,30). Thus, the stimuli differed from those in the present study in terms of their meaningfulness and degree of abstract symbolism.…”

“…On the other hand, sign language experience appears to drive a specialization of these regions for signs over nonsigns in signers, whereas similar levels of activation are seen for gestures and spoken descriptions of human actions in nonsigners (27). In the right hemisphere, when viewing gestures both signers and nonsigners show activation of the homologs of the left hemisphere language regions noted above, including the STS (the STSp associated with biological motion, as well as more anterior areas), inferior and superior parietal regions, and the IFG.…”

Does comprehension of symbolic gestures and corresponding-in-meaning words make use of motor simulation?

“…In accordance with previous work [5,24] a fronto-parietal circuit related to language or better to linking meaning to symbols in a modalityindependent way may be used for comprehension of symbolic gesture and corresponding word. In contrast, motor circuits including primary motor area are likely activated to comprehend action words used in actual and even metaphoric context [8,25].…”