Spatial and Linguistic Aspects of Visual Imagery in Sentence Comprehension

Abstract

There is mounting evidence that language comprehension involves the activation of mental imagery of the content of utterances (Barsalou, 1999;Bergen, Chang, & Narayan, 2004;Bergen, Narayan, & Feldman, 2003;Narayan, Bergen, & Weinberg, 2004;Richardson, Spivey, McRae, & Barsalou, 2003;Stanfield & Zwaan, 2001;Zwaan, Stanfield, & Yaxley, 2002). This imagery can have motor or perceptual content. Three main questions about the process remain under-explored, however. First, are lexical associations with perception or motion sufficient to yield mental simulation, or is the integration of lexical semantics into larger structures, like sentences, necessary? Second, what linguistic elements (e.g., verbs, nouns, etc.) trigger mental simulations? Third, how detailed are the visual simulations that are performed? A series of behavioral experiments address these questions, using a visual object categorization task to investigate whether up-or down-related language selectively interferes with visual processing in the same part of the visual field (following Richardson et al., 2003). The results demonstrate that either subject nouns or main verbs can trigger visual imagery, but only when used in literal sentences about real space-metaphorical language does not yield significant effects-which implies that it is the comprehension of the sentence as a whole and not simply lexical associations that yields imagery effects. These studies also show that the evoked imagery contains detail as to the part of the visual field where the described scene would take place.

Visual motion aftereffect from understanding motion language

“…A number of findings suggest that people do spontaneously engage in imagery during language comprehension, and that processing language affects performance in subsequent perceptual tasks (1)(2)(3)(4)(5)(6)(7)(8)(9).…”

Mental Imagery in Translation Processes

“…These approaches are supported by strong experimental evidence, for instance, on the details of visual simulations (e.g., Stanfield/Zwaan 2001;Richardson et al 2003;Bergen et al 2007) and motor simulations (e.g., Glenberg/Kaschak 2002;Taylor/Zwaan 2001). One of these approaches, the Language and Situated Simulation (LASS) theory Simmons et al 2008), is particularly relevant to the purposes of the present study, since it assumes that knowledge is represented by multiple systems, which is coherent with the theoretical background of the OSIVQ.…”

Abstract Concepts in Grounded Cognition

“…In subsequent studies showed that positive words direct attention upwards and negative words direct attention downward (similar to related effects in concrete domains, e.g., Bergen et al, 2007;Estes et al, 2008;Richardson et al, 2003). People were faster to indicate whether a letter presented on the top of the screen was a p or a q after evaluating a positive word, than after evaluating a negative word.…”

“…The critical difference between interference and facilitation seems to be the length of time between the presentation of the linguistic stimulus and the perceptual stimulus. If the visual brain areas are still busy representing the linguistic stimulus (e.g., between 50 to 200 milliseconds) then interference is more likely, but after the visual brain areas are no longer active, the previous visual representation of vertical or horizontal locations might facilitate recognition (for a discussion, see Bergen et al, 2007).…”