Modality-specific dysfunctional neural processing of social-abstract and non-social-concrete information in schizophrenia

Abstract

Background: Schizophrenia (SZ) is characterized by marked social dysfunctions encompassing potential deficits in the processing of social and non-social information, especially in everyday settings where multiple modalities are present. To date, the neurobiological basis of these deficits remains elusive.Methods: In a functional magnetic resonance imaging (fMRI) study, 17 patients with schizophrenia or schizoaffective disorder, and 18 matched controls watched videos of an actor speaking, gesturing (unimodal), and both speaking and gesturing (bimodal) about social or non-social events in a naturalistic way. Participants had to judge whether each video contains person-related (social) or object-related (non-social) information.Results: When processing social content, controls activated the medial prefrontal cortex (mPFC) for both speech and gesture conditions; patients, in comparison to controls, showed no different activation in the speech condition but reduced activation in the mPFC in the gesture condition. For non-social content, across modalities, controls recruited the bilateral pre/postcentral gyrus, superior temporal gyrus, and insula, as well as the left occipitotemporal cortex; patients showed reduced activation of the left postcentral gyrus and the right insula only in the speech condition. Moreover, in the bimodal conditions, patients displayed improved task performance and comparable activation to controls in both social and non-social content. Conclusions:Patients with SZ displayed modality-specific aberrant neural processing of social and non-social information, which is not present for the bimodal conditions. This finding provides novel insights into dysfunctional social cognition in SZ, and may have potential therapeutic implications.

TMS reveals a two-stage priming circuit of gesture-speech integration

“…Here, we defined two stages of gesture-speech interaction: semantic priming of gestures onto the phonological processing of speech and unification of gesture and speech semantics to form context-appropriate semantic representations. The two processing stages in gesture-speech integration proposed in the present study are consistent with previous findings (Drijvers et al, 2018;He et al, 2018). In a study by (Kelly et al, 2004), when information contained in the preceding gesture and the following speech word was incongruent, there were early N1-P1 and P2 sensory effects followed by an N400 semantic effect; this was interpreted to indicate that gesture influenced word comprehension first at the level of sensory/phonological processing and later at the level of semantic processing.…”

The processing of semantic complexity and co-speech gestures in schizophrenia: a naturalistic, multimodal fMRI study

“…A similar facilitative effect of gesture when processing a story has been observed for elder vs. younger participants, i.e., group difference was reduced when gestures were presented together with story segments (49). In line with these findings, for isolated sentences, despite the fact that patients showed neural aberrance when processing abstract-social gestures and concrete-non-social speech, group difference was significantly reduced when these events were presented in a multimodal form (28). In the current study, we extend prior findings by showing a similar facilitative nature of gesture during speech comprehension: In a naturalistic story setting, even if patients neural aberrance may be more likely to derive from their impaired semantic prediction and working memory capacity, which are not as heavily demanded in isolated situations, nevertheless, a multimodal context is, again, proven beneficial (6,21).…”

“…For example, when processing gestures in an abstract sentence context (metaphoric gestures), patients showed impaired behavioral performance to match the semantic information of gesture to a corresponding sentence (26,27), even more so for those with more severe positive formal thought disorders (17). Of note, even if the processing of abstract gestures is impaired in schizophrenia (18,28), when the same information is presented bimodally (speech accompanied by gesture), fMRI results suggest that patients showed comparable brain activations in comparison to controls (28). These prior studies, together, may suggest that patients' processing of gesture and its semantics may benefit from a multimodal context, as they would mostly encounter in daily naturalistic situations.…”