The influence of state change on object representations in language comprehension

Abstract

To understand language people form mental representations of described situations. Linguistic cues are known to influence these representations. In the present study, participants were asked to verify whether the object presented in a picture was mentioned in the preceding words. Crucially, the picture either showed an intact original state or a modified state of an object. Our results showed that the end state of the target object influenced verification responses. When no linguistic context was provided, participants responded faster to the original state of the object compared to the changed state (Experiment 1). However, when linguistic context was provided, participants responded faster to the modified state when it matched, rather than mismatched, the expected outcome of the described event (Experiment 2 and Experiment 3). Interestingly, as for the original state, the match/mismatch effects were only revealed after reading the past tense (Experiment 2) sentences but not the future-tense sentences (Experiment 3). Our findings highlight the need to take account of the dynamics of event representation in language comprehension that captures the interplay between general semantic knowledge about objects and the episodic knowledge introduced by the sentential context.

The activation of object-state representations during online language comprehension

“…"weigh"). Using the picture verification paradigm, Kang, Eerland, Joergensen, Zwaan, & Altmann (2019) found that participants reacted faster to a target probe object (e.g. a dropped ice cream cone) when it matched the end state of the event ("The woman will drop the ice cream") than when it (most likely) did not ("The woman will choose the ice cream").…”

“…However, they might also, or instead, retrieve the object representation that matches the most probable end state as described in linguistic context (e.g. more looks at the intact state when a minimal change of state is described, and more looks at the chopped state when a substantial change is described), as might be predicted by Kang et al (2019).…”

“…Our primary interests in the current experiment were twofold: First, to explore whether listeners display a bias in the first sentence to look towards the object-state that matches the described end state of the target object (as predicted by Kang et al, 2019), or whether they display a bias to look towards the initial state of the target object, corresponding to the state that would be acted upon in the first sentence (as predicted by Altmann & Kamide, 1999. Second, to establish the time course with which attention in the second sentence towards the depictions of the different target states is modulated by the interaction between Temporal Context (But first vs.…”

Individual Differences in Keeping Track of Object-state Representations during Language Comprehension: Evidence from Eye Movements

“…Visual attention, a core cognitive process that is vital for our survival, is welldocumented to be modulated by the language. Upon hearing or reading an object name or an event, we activate implied perceptual properties of the target object such as its state (Kang et al, 2019), orientation (Stanfield & Zwaan, 2001), shape (Zwaan, Stanfield, & Yaxley, 2002), visibility (Yaxley & Zwaan, 2007), color (Hoeben-Mannaert , Dijkstra, & Zwaan, 2017;; Richter & Zwaan, 2009), and location and motion direction (Bergen, Lindsay, Matlock, & Narayanan, 2007;; Chasteen, Burdzy, & Pratt, 2010;; Dudschig, Souman, Lachmair, Vega, & Kaup, 2013;; Estes, Verges, & Adelman, 2015;; Estes, Verges, Barsalou, 2008;; Gozli, Chasteen, & Pratt, 2013;; Richardson, Spivey, Barsalou, & McRae, 2003;; Šetić & Domijan, 2007).…”

“…We predict that participants would establish multiple conflicting object-state representations during sentence comprehension and cognitive resources are required to resolve this ambiguity. During the first sentence, participants may establish the described mental representations of the target object and may pay more attention to the item when it matches than mismatches the described object representation (Kang et al, 2019(Kang et al, , 2020. During the second sentence, participants may update the mental representation of the target object by suppressing or discarding the irrelevant ones (Hindy, Kalenik, & Thompson-Schill, 2012;; Solomon, Hindy, Altmann, & Thompson-Schill, 2015).…”