The contents of predictions in sentence comprehension: Activation of the shape of objects before they are referred to

Abstract

When comprehending concrete words, listeners and readers can activate specific visual information such as the shape of the words’ referents. In two experiments we examined whether such information can be activated in an anticipatory fashion. In Experiment 1, listeners’ eye movements were tracked while they were listening to sentences that were predictive of a specific critical word (e.g., “moon” in “In 1969 Neil Armstrong was the first man to set foot on the moon”). 500 ms before the acoustic onset of the critical word, participants were shown four-object displays featuring three unrelated distractor objects and a critical object, which was either the target object (e.g., moon), an object with a similar shape (e.g., tomato), or an unrelated control object (e.g., rice). In a time window before shape information from the spoken target word could be retrieved, participants already tended to fixate both the target and the shape competitors more often than they fixated the control objects, indicating that they had anticipatorily activated the shape of the upcoming word's referent. This was confirmed in Experiment 2, which was an ERP experiment without picture displays. Participants listened to the same lead-in sentences as in Experiment 1. The sentence-final words corresponded to the predictable target, the shape competitor, or the unrelated control object (yielding, for instance, “In 1969 Neil Armstrong was the first man to set foot on the moon/tomato/rice”). N400 amplitude in response to the final words was significantly attenuated in the shape-related compared to the unrelated condition. Taken together, these results suggest that listeners can activate perceptual attributes of objects before they are referred to in an utterance

Putting things in new places: Linguistic experience modulates the predictive power of placement verb semantics

“…There is also evidence that visual properties of an object are anticipated (e.g., shape; Rommers, Meyer, Praamstra, & Huettig, 2013). Generating predictions is considered crucial to the language processing system, leading to faster and more efficient mental operations (e.g., Farmer, Brown, & Tanenhaus, 2013;Fine, Jaeger, Farmer, & Qian, 2013;Hale, 2003;Levy, 2008).…”

Representing composed meanings through temporal binding

“…To explain this discrepancy, Experiment 1c suggests that the composition advantage is due to predictive processes that are outside the purview of models such as DORA and LISA. In particular, participants appear able to rapidly transform entirely novel compositional concepts into accurate predictions about visual stimuli (see also Rommers et al, 2013;Zwaan et al, 2002): When predictive strength is matched between one-word and two-word stimuli, participants perform in a manner consistent with the predictions of models such as DORA and LISA.…”

“…This difference could perhaps explain the composition advantage: Participants could more easily predict the correct picture for pink tree (pink trees at one of three orientations) than for tree (trees of six colours at one of three orientations). This explanation is given some prima facie plausibility by recent demonstrations that participants can rapidly translate linguistic information into predictions about the likely visual form of a referent (Rommers, Meyer, Praamstra, & Huettig, 2013;Zwaan, Stanfield, & Yaxley, 2002). However there is also an important reason for doubting it: there was no need for participants in our one-word condition to even attend to the color of the subsequent picture, and it is known that the shape of an object can be processed separately from its color (Garner & Felfoldy, 1970).…”

Revisiting the looking at nothing phenomenon: Visual and semantic biases in memory search

“…In the crucial displays, some objects in the display have a relationship with a specific word in the spoken utterance. Results show that people spend more time fixating related than unrelated objects, whether this relationship is semantic (e.g., Huettig & Altmann, 2005;Yee & Sedivy, 2006;Yee, Overton, & Thompson-Schill, 2009) or visual in nature (e.g., Dahan & Tanenhaus, 2005;Dunabeitia, Aviles, Afonso, Scheepers, & Carreiras, 2009;Huettig & Altmann, 2007;Huettig & Altmann, 2011;Rommers, Meyer, & Huettig, 2015;Rommers, Meyer, Praamstra, & Huettig, 2013). In de Groot, Huettig, and Olivers (2016), we directly compared visual and semantic biases in the visual search and the visual world paradigm.…”