The activation of modality-specific representations during discourse processing

Abstract

Previous research has shown that readers generate mental images of events. Most studies have investigated imagery during the reading of short texts, which also included explicit judgment tasks. In two fMRI studies, we assessed whether modality-specific imagery occurs during naturalistic, discourse comprehension. We identified clauses in the texts that elicited auditory, motor, or visual imagery. In both studies, reading motor imagery clauses was associated with increases in activity in left postcentral and precentral sulci, and reading auditory imagery clauses was associated with increases in left superior temporal gyrus and perisylvian language-related regions. Study 2 compared presentation of connected discourse to a condition in which unconnected sentences were presented, preventing the establishment of global coherence. Sensorimotor imagery was strongest when readers were able to generate a globally coherent discourse representation. Overall, these results suggest that modality-specific imagery occurs during discourse comprehension and it is dependent on the development of discourse-level representations.

Grounding language in the neglected senses of touch, taste, and smell

“…The evidence for the automatic mental simulation of touch is perhaps stronger, with texture regions of the brain activated when reading and understanding texture-related sentences (Lacey et al, 2012). In order to critically assess the automaticity of mental simulations, future studies could focus on the time-course of activations (e.g., Ostarek & Vigliocco, 2017), or use tasks less likely to elicit strategic mental imagery and more akin to everyday language use, such as narrative comprehension (Kurby & Zacks, 2013;Willems & van Gerven, 2018).…”

Three symbol ungrounding problems: Abstract concepts and the future of embodied cognition

“…Similarly, motor-evoked potentials have been found to increase for first-person but not for third-person action verbs (Papeo, Corradi-Dell'Acqua, & Rumiati, 2011). Coming at the issue of flexibility from a different angle, there is evidence that larger narratives can selectively engage grounded representations (Chow et al, 2014;Kurby & Zacks, 2013).…”

Semantic Processing

“…A few studies of sound-related knowledge (Goldberg et al, 2006;Kellenbach et al, 2001;Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008;Kurby & Zacks, 2013) reported activation in or near the posterior superior temporal sulcus (STS), an auditory association region implicated in environmental sound recognition (Leaver & Rauschecker, 2010;Lewis et al, 2004). A few studies of sound-related knowledge (Goldberg et al, 2006;Kellenbach et al, 2001;Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008;Kurby & Zacks, 2013) reported activation in or near the posterior superior temporal sulcus (STS), an auditory association region implicated in environmental sound recognition (Leaver & Rauschecker, 2010;Lewis et al, 2004).…”