Perceived communicative intent in gesture and language modulates the superior temporal sulcus

Abstract

Behavioral evidence and theory suggest gesture and language processing may be part of a shared cognitive system for communication. While much research demonstrates both gesture and language recruit regions along perisylvian cortex, relatively less work has tested functional segregation within these regions on an individual level. Additionally, while most work has focused on a shared semantic network, less has examined shared regions for processing communicative intent. To address these questions, functional and structural MRI data were collected from 24 adult participants while viewing videos of an experimenter producing communicative, Participant-Directed Gestures (PDG) (e.g., "Hello, come here"), noncommunicative Self-adaptor Gestures (SG) (e.g., smoothing hair), and three written text conditions: (1) Participant-Directed Sentences (PDS), matched in content to PDG, (2) Third-person Sentences (3PS), describing a character's actions from a third-person perspective, and (3) meaningless sentences, Jabberwocky (JW). Surface-based conjunction and individual functional region of interest analyses identified shared neural activation between gesture (PDGvsSG) and language processing using two different language contrasts. Conjunction analyses of gesture (PDGvsSG) and Third-person Sentences versus Jabberwocky revealed overlap within left anterior and posterior superior temporal sulcus (STS). Conjunction analyses of gesture and Participant-Directed Sentences to Third-person Sentences revealed regions sensitive to communicative intent, including the left middle and posterior STS and left inferior frontal gyrus. Further, parametric modulation using participants' ratings of stimuli revealed sensitivity of left posterior STS to individual perceptions of communicative intent in gesture. These data highlight an important role of the STS in processing participant-directed communicative intent through gesture and language. Hum Brain Mapp 37:3444-3461, 2016. © 2016 Wiley Periodicals, Inc.

Widespread and lateralized social brain activity for processing dynamic facial expressions

“…This result suggests the presence of functionally lateralized and nonlateralized subregions in the STS region. This idea is consistent with data showing functionally segregated STS subregions that process social signals (Engell & Haxby, 2007;Pelphrey et al, 2005;Redcay et al, 2016;Schobert et al, 2018;Wheaton et al, 2004). The right-dominant activity in the IFG and occipital region corroborates the findings of some previous studies although they did not statistically test for lateralized activity (e.g., .…”

Perceiving social interactions in the posterior superior temporal sulcus

“…If humans have a similarly selective cortical response to social interactions, where might it be found in the brain? One region that seems a likely prospect for such a response is the pSTS, which has been previously shown to respond during the perception of a wide variety of socially significant stimuli, including biological motion (18), dynamic faces (19), direction of gaze (20), emotional expressions (21), goal-directed actions (22), and communicative intent (23).…”

Gesture’s body orientation modulates the N400 during semantic integration of gesture and visual sentence

“…These caveats have been addressed by previous studies using fMRI. For example, in Straube et al (2010), a direct comparison between frontal and lateral co-speech gestures activated the mPFC and other regions in the mentalizing network, thus indicating the potential social-cues conveyed by body orientation (see also Nagels et al, 2015;Redcay et al, 2016;Saggar et al, 2014). However, for these communicative gestures, in the EEG domain, whether body orientation affects sensorimotor alpha or beta oscillations remains unclear.…”