Supramodal neural processing of abstract information conveyed by speech and gesture

Abstract

Abstractness and modality of interpersonal communication have a considerable impact on comprehension. They are relevant for determining thoughts and constituting internal models of the environment. Whereas concrete object-related information can be represented in mind irrespective of language, abstract concepts require a representation in speech. Consequently, modality-independent processing of abstract information can be expected. Here we investigated the neural correlates of abstractness (abstract vs. concrete) and modality (speech vs. gestures), to identify an abstractness-specific supramodal neural network. During fMRI data acquisition 20 participants were presented with videos of an actor either speaking sentences with an abstract-social [AS] or concrete-object-related content [CS], or performing meaningful abstract-social emblematic [AG] or concrete-object-related tool-use gestures [CG]. Gestures were accompanied by a foreign language to increase the comparability between conditions and to frame the communication context of the gesture videos. Participants performed a content judgment task referring to the person vs. object-relatedness of the utterances. The behavioral data suggest a comparable comprehension of contents communicated by speech or gesture. Furthermore, we found common neural processing for abstract information independent of modality (AS > CS ∩ AG > CG) in a left hemispheric network including the left inferior frontal gyrus (IFG), temporal pole, and medial frontal cortex. Modality specific activations were found in bilateral occipital, parietal, and temporal as well as right inferior frontal brain regions for gesture (G > S) and in left anterior temporal regions and the left angular gyrus for the processing of speech semantics (S > G). These data support the idea that abstract concepts are represented in a supramodal manner. Consequently, gestures referring to abstract concepts are processed in a predominantly left hemispheric language related neural network.

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

“…5B, right), outside the region where emblems differ from pantomimes. Weak effects of abstractness/ concreteness are compatible with current results showing comparable activity in the lateral temporal cortex for abstract and concrete words (Bedny et al, 2008;Hernández et al, 2014;, and abstractness/concreteness effects in regions outside this territory (Binder et al, 2009;Straube et al, 2013). In summary, although emblems and pantomimes elicited comparable activity in the largest part of left and right MTG, they differed in two clusters.…”

Atypical MEG inter-subject correlation during listening to continuous natural speech in dyslexia

“…In terms of phoneme processing, the left middle temporal cortex would be expected to play a major role, as it is an integral part of speech and word processing (Hickok and Poeppel, 2007). In fMRI studies, the peak location for differences between our groups found for the beta band has been frequently associated with activations during listening to speech in various ways (Narain et al, 2003;Oechslin et al, 2010;Straube et al, 2013b;Nagels et al, 2013;Evans et al, 2016, Wolf et al, 2017.…”

Modelo de conectividad de la circunvolución angular en el lenguaje: metaanálisis de neuroimágenes funcionales

“…Otros hallazgos provenientes de estudios de neuroimagen funcional describen que, mientras que la integración auditivovisual está más relacionada con la región angular derecha [31], la región angular izquierda parece estar más involucrada en redes de procesamiento abstracto independiente de la modalidad sensorial [32]. De hecho, Fairhall y Caramazza [33] proponían que la región angular izquierda era parte de una red intermodal que intervenía en la representación conceptual de los objetos.…”