Simulating visibility during language comprehension

Yaxley, Richard H.; Zwaan, Rolf A.

doi:10.1016/j.cognition.2006.09.003

Cited by 99 publications

(86 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Zwaan, Stanfield, and Yaxley (2002), for example, found that after reading a sentence such as The ranger saw the eagle in the sky participants were faster to judge a picture of an eagle as mentioned versus not mentioned in the sentence when the depicted shape (wings outstretched) matched the shape implied in the sentence (the eagle is in the sky) compared to when it did not match (perched). Stanfield and Zwaan (2001) observed related findings concerning object orientation, while Yaxley and Zwaan (2005) provided evidence that readers simulate even the visibility of described scenes (e.g., as foggy versus clear). The above observations have generally been taken as support for the seamless interaction of visual and linguistic representations (Tanenhaus et al, 1995), on the one hand, and for multi-modal sensorimotor-derived meaning representations on the other (Barsalou, 1999b).…”

Section: Introductionmentioning

confidence: 88%

Situated sentence processing: The coordinated interplay account and a neurobehavioral model

2010

View full text Add to dashboard Cite

Empirical evidence demonstrating that sentence meaning is rapidly reconciled with the visual environment has been broadly construed as supporting the seamless interaction of visual and linguistic representations during situated comprehension. Based on recent behavioral and neuroscientific findings, however, we argue for the more deeply rooted coordination of the mechanisms underlying visual and linguistic processing, and for jointly considering the behavioral and neural correlates of scene-sentence reconciliation during situated comprehension. The Coordinated Interplay Account (CIA; Knoeferle & Crocker, 2007) asserts that incremental linguistic interpretation actively directs attention in the visual environment, thereby increasing the salience of attended scene information for comprehension. We review behavioral and neuroscientific findings in support of the CIA's three processing stages: (i) incremental sentence interpretation, (ii) language-mediated visual attention, and (iii) the on-line influence of non-linguistic visual context. We then describe a recently developed connectionist model which both embodies the central CIA proposals and has been successfully applied in modeling a range of behavioral findings from the visual world paradigm (Mayberry, Crocker, & Knoeferle, in press). Results from a new simulation suggest the model also correlates with event-related brain potentials elicited by the immediate use of visual context for linguistic disambiguation (Knoeferle, Habets, Crocker, & Münte, 2008). Finally, we argue that the mechanisms underlying interpretation, visual attention, and scene apprehension are not only in close temporal synchronization, but have co-adapted to optimize real-time visual grounding of situated spoken language, thus facilitating the association of linguistic, visual and motor representations that emerge during the course of our embodied linguistic experience in the world.

show abstract

Section: Introductionmentioning

confidence: 88%

Situated sentence processing: The coordinated interplay account and a neurobehavioral model

2010

View full text Add to dashboard Cite

show abstract

“…Recent embodied and simulation based approaches to language comprehension predict that comprehension of language triggers mental simulation of events and objects as described in the language [1][2][3][4]. According to theories of embodied cognition [5], such simulation depends upon the mental representations that were formed during comprehenders' actual perceptual experience and interaction with the environment.…”

Section: Introductionmentioning

confidence: 99%

Simulating Motion in Figurative Language Comprehension

Singh¹,

Mishra²

2010

Open Neuroimag J

View full text Add to dashboard Cite

Abstract:In this visual world eye tracking study we explored simulation of fictive motion during language comprehension in figurative sentences in Hindi. Eye movement measures suggest that language comprehenders gaze longer at visual scenes on hearing fictive motion sentences compared to their literal counterparts. The results support previous findings in English and provide cross linguistic evidence for the simulation and embodied views of language processing. We discuss the findings in the light of neuroimaging models and language vision interaction.

show abstract

“…Support for this idea is extensive, indicating that readers simulate multiple aspects of the characters' experience, such as their visual perspective, including the shape, orientation, and color of objects (Connell, 2007;Stanfield & Zwaan, 2001;Yaxley & Zwaan, 2007), as well as their motor movements (Speer, Reynolds, Swallow, & Zachs, 2009;Taylor, Lev-Ari, & Zwaan, 2008;. For example, Glenberg and Kaschak (2002) found that after reading a sentence describing a character making a movement in a particular direction (e.g., "He closed the drawer"), readers were faster to press a key on a response box that involved making a movement in the same direction as the character (e.g., moving their hand away from their body) than making a movement in the opposite direction.…”

mentioning

confidence: 99%

When story characters communicate: readers’ representations of characters’ linguistic exchanges

Drumm

Klin

2011

Mem Cogn

View full text Add to dashboard Cite

Klin, Ralano, and Weingartner (2007) found transfer effects when a phrase, described as part of a note one character had left for another, was repeated across two passages. However, when the phrase was part of a note in story A and part of a conversation in story B, transfer effects were eliminated (Klin & Drumm, 2010). Klin and Drumm concluded that readers encoded the perceptual features of story characters' linguistic exchanges and that the mismatch (visual vs. auditory) eliminated transfer effects. The present experiments support this conclusion and also demonstrate that readers encode details of the social interaction that surrounds the characters' linguistic exchanges: Effects were reduced when the phrase in story A was part of a direct social interaction between the characters (e.g., phone conversation), whereas in story B, the interaction was indirect (e.g., voicemail). More generally, readers are exquisitely tuned to subtle aspects of characters' linguistic exchanges.

show abstract

Simulating visibility during language comprehension

Cited by 99 publications

References 12 publications

Situated sentence processing: The coordinated interplay account and a neurobehavioral model

Situated sentence processing: The coordinated interplay account and a neurobehavioral model

Simulating Motion in Figurative Language Comprehension

When story characters communicate: readers’ representations of characters’ linguistic exchanges

Contact Info

Product

Resources

About