Toward Semantics in the Wild: Activation to Manipulable Nouns in Naturalistic Reading

Abstract

The neural basis of language processing, in the context of naturalistic reading of connected text, is a crucial but largely unexplored area. Here we combined functional MRI and eye tracking to examine the reading of text presented as whole paragraphs in two experiments with human subjects. We registered high-temporal resolution eye-tracking data to a low-temporal resolution BOLD signal to extract responses to single words during naturalistic reading where two to four words are typically processed per second. As a test case of a lexical variable, we examined the response to noun manipulability. In both experiments, signal in the left anterior inferior parietal lobule and posterior inferior temporal gyrus and sulcus was positively correlated with noun manipulability. These regions are associated with both action performance and action semantics, and their activation is consistent with a number of previous studies involving tool words and physical tool use. The results show that even during rapid reading of connected text, where semantics of words may be activated only partially, the meaning of manipulable nouns is grounded in action performance systems. This supports the grounded cognition view of semantics, which posits a close link between sensory-motor and conceptual systems of the brain. On the methodological front, these results demonstrate that BOLD responses to lexical variables during naturalistic reading can be extracted by simultaneous use of eye tracking. This opens up new avenues for the study of language and reading in the context of connected text.

Fluent Speakers of a Second Language Process Graspable Nouns Expressed in L2 Like in Their Native Language

“…Two recent studies (Marino et al, 2014 ; Zhang et al, 2016 ) showed a similar modulation of the motor system during the processing of objects and nouns belonging to the same category. Additionally in an fMRI study (Desai et al, 2016 ), during reading nouns expressing graspable objects activations were found in areas also involved in action performance, thus supporting a grounded view of semantics. Taken as a whole, current literature supports the notion that processing visually presented graspable objects and nouns referring to the same object category recruit common neural substrates crucially involving the motor system (Ganis et al, 1996 ; Vandenberghe et al, 1996 ).…”

A Confluence of Complexity: Intersections Among Reading Theory, Neuroscience, and Observations of Young Readers

“…In addition to decoding letters and words, reading involves readers’ experiences, including multimodal integration of linguistic and nonlinguistic knowledge involved in meaning construction. Early evidence suggests that “experiential and linguistically acquired knowledge can be detected in brain activity elicited in reading natural sentences” (Anderson et al, 2019, p. 8969), in contrast to reading lists of words or pseudowords (Desai, Choi, Lai, & Henderson, 2016). In short, naturalistic reading activates the same sensorimotor systems as nonlinguistic experiences.…”

The Embodied Penman: Effector‐Specific Motor–Language Integration During Handwriting

“…At the same time, our study carries an important methodological implication for handwriting studies in general: Given that fine‐grained semantic aspects of the target words can modulate writing kinematics, it seems crucial to control the ratio of action‐to‐non‐action words in an experiment’s stimulus sets, as gross outcomes could be partially driven by inconspicuous embodied effects differing between conditions. Moreover, this consideration could be contemplated in future studies extending our paradigm beyond the single‐word level, so as to explore language‐embodiment effects on more realistic linguistic materials (Desai et al, ; García et al, ; Trevisan et al, ).…”