Preferential Inspection of Recent Real-World Events Over Future Events: Evidence from Eye Tracking during Spoken Sentence Comprehension

Abstract

Eye-tracking findings suggest people prefer to ground their spoken language comprehension by focusing on recently seen events more than anticipating future events: When the verb in NP1-VERB-ADV-NP2 sentences was referentially ambiguous between a recently depicted and an equally plausible future clipart action, listeners fixated the target of the recent action more often at the verb than the object that hadn’t yet been acted upon. We examined whether this inspection preference generalizes to real-world events, and whether it is (vs. isn’t) modulated by how often people see recent and future events acted out. In a first eye-tracking study, the experimenter performed an action (e.g., sugaring pancakes), and then a spoken sentence either referred to that action or to an equally plausible future action (e.g., sugaring strawberries). At the verb, people more often inspected the pancakes (the recent target) than the strawberries (the future target), thus replicating the recent-event preference with these real-world actions. Adverb tense, indicating a future versus past event, had no effect on participants’ visual attention. In a second study we increased the frequency of future actions such that participants saw 50/50 future and recent actions. During the verb people mostly inspected the recent action target, but subsequently they began to rely on tense, and anticipated the future target more often for future than past tense adverbs. A corpus study showed that the verbs and adverbs indicating past versus future actions were equally frequent, suggesting long-term frequency biases did not cause the recent-event preference. Thus, (a) recent real-world actions can rapidly influence comprehension (as indexed by eye gaze to objects), and (b) people prefer to first inspect a recent action target (vs. an object that will soon be acted upon), even when past and future actions occur with equal frequency. A simple frequency-of-experience account cannot accommodate these findings.

Language comprehension in rich non-linguistic contexts: combining eye-tracking and event-related brain potentials

“…In this situation, listeners rapidly inspected the target of the recent action (e.g., a candelabra that had been polished) in preference to the target of a future polishing action (e.g., polishing crystal glasses, target-condition assignment was counterbalanced, Knoeferle & Crocker, 2007). The recent-event preference replicated with real-world events (Knoeferle, Carminati, Abashidze, & Essig, 2011a) and when the within-experiment frequency of future (relative to recent) actions was increased to 75 (vs. 25) percent (Abashidze, Knoeferle, & Carminati, 2013). …”

Losing track of time? Processing of time reference inflection in agrammatic and healthy speakers of German

“…In their study with NBDs, Altmann and Kamide (2007) used few change-of-state verbs in their materials and, moreover, they showed pictures of two different objects, for example, a wine glass and a beer glass, so that no prototypical past picture was included. However, Knoeferle et al (2011) found that locations of recently seen events attract more eye gazes during verb processing than potential target locations of future events. This finding seems similar to the pastpicture preference observed in our study.…”

“…Altman and Kamide thus demonstrated that when healthy speakers of English interpret the time reference of a verb, they direct their gaze towards an object that is anticipated as the grammatical theme. Also in real world events (with participants watching events acted out by the experimenter), it has been found that time reference guides anticipatory eye movements to the location of the grammatical theme; however, gaze patterns between sentences referring to the past versus the future differed (Knoeferle et al, 2011). Participants listened to German sentences in either simple past with past time reference or simple present with future time reference as in (1) while inspecting a scene with two objects.…”

Priming Younger and Older Adults’ Sentence Comprehension: Insights from Dynamic Emotional Facial Expressions and Pupil Size Measures

“…Overall, findings show that visual context information exerts a very rapid effect on language processing and influences, for instance, the resolution of syntactically ambiguous sentences [1,2]. Findings also show that information such as an object's size, color [3], or shape [4], depicted clipart events [2], real-world action events [5], action affordances [6], and the spatial location of objects [7] are all rapidly integrated during sentence comprehension and can affect a listener's visual attention within a few hundred milliseconds (for a recent review, see [8]). …”