Modulation of BOLD Response in Motion-sensitive Lateral Temporal Cortex by Real and Fictive Motion Sentences

Abstract

Can linguistic semantics affect neural processing in feature-specific visual regions? Specifically, when we hear a sentence describing a situation that includes motion, do we engage neural processes that are part of the visual perception of motion? How about if a motion verb was used figuratively, not literally? We used fMRI to investigate whether semantic content can “penetrate” and modulate neural populations that are selective to specific visual properties during natural language comprehension. Participants were presented audiovisually with three kinds of sentences: motion sentences (“The wild horse crossed the barren field.”), static sentences, (“The black horse stood in the barren field.”), and fictive motion sentences (“The hiking trail crossed the barren field.”). Motion-sensitive visual areas (MT+) were localized individually in each participant as well as face-selective visual regions (fusiform face area; FFA). MT+ was activated significantly more for motion sentences than the other sentence types. Fictive motion sentences also activated MT+ more than the static sentences. Importantly, no modulation of neural responses was found in FFA. Our findings suggest that the neural substrates of linguistic semantics include early visual areas specifically related to the represented semantics and that figurative uses of motion verbs also engage these neural systems, but to a lesser extent. These data are consistent with a view of language comprehension as an embodied process, with neural substrates as far reaching as early sensory brain areas that are specifically related to the represented semantics.

Complementarity of Space and Time in Distance Representations

“…99-103). It seems that the results obtained in psycholinguistic experiments (Matlock, 2004b(Matlock, , 2006Matlock & Richardson, 2004;Richardson & Matlock, 2007), and brain studies (Wallentin et al, 2005;Saygin et al, 2010;Cacciari et al, 2011) demonstrate the fictive mode, in which processing of fictive motion takes place in a manner somewhat parallel to actual motion. This mode has arguably a greater potential for denoting spatial extension in terms of duration, since it evokes an association with physical movement.…”

“…Activation in motion sensitive visual areas of the brain observed for fictive motion may be attributed to mental scanning of the scenarios conveyed by fictive motion sentences. A more recent fMRI study (Saygin, McCullough, Alac & Emmorey, 2010) investigated activation of motion-sensitive visual brain areas during audiovisual presentation of fictive motion sentences. It found that fictive motion sentences engage motion-sensitive visual areas, however, to a lesser extent than actual motion sentences.…”

The neural basis of conceptualizing the same action at different levels of abstraction

“…But they are more frequently regarded as forming a subset of the functional network thought to enable representation of the visual and somatomotor features of actions when they are perceived, performed, conceptualized, and verbally processed (Caspers et al, 2010;Kemmerer et al, 2012;Molenberghs et al, 2012;Pulvermuller, 2013;Watson et al, 2013;Rizzolatti et al, 2014;Urgesi et al, 2014;Kemmerer, 2015). This included a region of the left posterior MTG that has been associated with encoding the visual motion components of action concepts (Chen et al, 2008;Deen and McCarthy, 2010;Saygin et al, 2010;Wallentin et al, 2011;Humphreys et al, 2013;Watson et al, 2013). Interestingly, this seems to contrast with the view that the left posterior MTG represents more schematic aspects of the event structures encoded by both action and non-action verbs/sentences (Bedny et al, 2008(Bedny et al, , 2012.…”

Visual motion aftereffect from understanding motion language

“…The explanation frequently offered is that the representations generated during the course of language comprehension share processing resources with perception, recruiting some of the very same brain regions (10). As evidence for this possibility, neuroimaging [functional MRI (fMRI)] measures have revealed that classically "perceptual" brain areas are recruited in service of language comprehension (11). Although these findings are consistent with the hypothesis, questions remain.…”