The activation of object-state representations during online language comprehension

Abstract

Understanding the time-course of event knowledge activation is crucial for theories of language comprehension. We report two experiments using the 'visual world paradigm' (VWP) that investigated the dynamic mapping between object-state representations and real-time language processing. In Experiment 1, participants heard sentences that described events resulting in either a substantial change of state (e.g. The chef will chop the onion) or a minimal change of state (e.g. The chef will weigh the onion). Concurrently, they viewed pictures depicting two versions of the target object (e.g., an onion) corresponding to the intact and changed states, and two unrelated distractors. A second sentence referred to the object with either a backward or a forward shift in event time (e.g. But first/And then, he will smell the onion). In Experiment 2, Degree of Change was manipulated by using different nouns in the first sentence (e.g. The girl will stomp on the penny/egg). The second sentence was similar to the ones used in Experiment 1 (e.g., But first/And then, she will look at the penny/egg). The results from both experiments showed that participants looked more at the 'appropriate' state of the object that matched the language context, but the shift of visual attention emerged only when the object name was heard. Our findings suggest that situationally appropriate object representations do trigger eye movements to the corresponding states of the target object, but inappropriate representations are not necessarily eliminated from consideration until the language forces it.

Processing focus in native and non-native speakers of English: an eye-tracking study in the visual world paradigm

“…The participants were not asked to give any behavioral responses such as pressing keys or clicking a mouse. The "look and listen" version of the visual world paradigm has been widely used in previous studies (e.g., Altmann & Kamide, 1999Kang et al, 2020;Salverda et al, 2011). In a task-oriented visual world paradigm, participants are instructed to move or point at the target entity.…”

Tracking Object-State Representations During Real-Time Language Comprehension by Native and Non-native Speakers of English

“…The sample size was determined based on a previous study (Kang et al, 2020). Compared to the previous study, the present study has fewer conditions (2 vs. 4) but more trials per condition than the previous study (12 vs. 9).…”

“…In the 2nd-9th time windows, we included Degree of Change, Object-state, their interaction as fixed effects, participants as random effects in bysubject models, and items as random effects in by-item models. 1 See example models below: By-subject<-lmer(Trans_Prop∼Degree-of-Change * Object -state + (1| Subject), data = T2) 1 We also analyzed the data using linear mixed models by including Object-state, Degree of Change, and their interactions as fixed effects, and both participants and items as random effects by following Kang et al (2020). During the 3rd time window, there was a significant interaction among native speakers (χ 2 = 5.70, p = 0.017), but not among non-native speakers (χ 2 = 0.037, p = 0.848).…”

“…According to the "intersecting object histories" (IOH) hypothesis, dynamic changes in objects across time are used as primitives of event representations (Altmann and Ekves, 2019). Multiple representations of objects are activated and updated during language processing (e.g., Hindy et al, 2012Hindy et al, , 2015Solomon et al, 2015;Kang et al, 2019Kang et al, , 2020Horchak and Garrido, 2020;Hupp et al, 2020;Lee and Kaiser, 2021;Misersky et al, 2021;Santin et al, 2021). Kang et al (2020) revealed that native speakers of English shifted their eye movements between two competing object-state representations of the target object in real-time language comprehension.…”

“…Multiple representations of objects are activated and updated during language processing (e.g., Hindy et al, 2012Hindy et al, , 2015Solomon et al, 2015;Kang et al, 2019Kang et al, , 2020Horchak and Garrido, 2020;Hupp et al, 2020;Lee and Kaiser, 2021;Misersky et al, 2021;Santin et al, 2021). Kang et al (2020) revealed that native speakers of English shifted their eye movements between two competing object-state representations of the target object in real-time language comprehension. In their study, participants were asked to listen to short stories in 2 × 2 conditions, such as "The chef will chop/weigh the onion.…”