World knowledge affects prediction as quickly as selectional restrictions: evidence from the visual world paradigm

Abstract

There has been considerable debate regarding the question of whether linguistic knowledge and world knowledge are separable and used differently during processing or not (Hagoort, Hald, Bastiaansen, & Petersson, 2004; Matsuki et al., 2011; Paczynski & Kuperberg, 2012; Warren & McConnell, 2007; Warren, McConnell, & Rayner, 2008). Previous investigations into this question have provided mixed evidence as to whether violations of selectional restrictions are detected earlier than violations of world knowledge. We report a visual-world eye-tracking study comparing the timing of facilitation contributed by selectional restrictions versus world knowledge. College-aged adults (n=36) viewed photographs of natural scenes while listening to sentences. Participants anticipated upcoming direct objects similarly regardless of whether facilitation was provided by only world knowledge or a combination of selectional restrictions and world knowledge. These results suggest that selectional restrictions are not available earlier in comprehension than world knowledge.

Concurrent use of animacy and event-knowledge during comprehension: Evidence from event-related potentials

“…These findings highlight the prominent role of animacy information in sentence processing. However, the extent to which animacy is prioritized over other forms of semantic information remains unclear, particularly when it is difficult to disentangle whether these effects could reflect processing of (im)plausibility (see also Milburn et al, 2016). Here, like PK12, we address this issue by investigating whether semantic relatedness has the same impact on how people process violations of animacy and those of real-world event-knowledge.…”

“…ERP results suggest that comprehenders rapidly assess the animacy of incoming arguments and its compatibility with selectional restrictions imposed by preceding verbs (Nieuwland & Van Berkum, 2005Paczynski & Kuperberg, 2012;Szewczyk & Schriefers, 2013). However, it is an open question whether or not animacy selection restrictions are prioritizedthat is, whether they are processed prior to and independently of other forms of semantic information, such as general real-world event knowledge that allows people to evaluate sentence plausibility (e.g., McRae & Matsuki, 2009;Milburn, Warren & Dickey, 2016;Nieuwland & Van Berkum, 2006).…”

“…Incremental processing enables comprehenders to rapidly realize when a word is semantically incompatible with prior context, a situation that leads to a processing cost relative to compatible words. Previous research has tried to elucidate whether certain types of semantic incompatibility between a word and its sentence context incur greater processing costs, or are more rapidly detected than others (e.g., Marslen-Wilson et al, 1988;Milburn et al, 2016;Staub, Rayner, Pollatsek, & Hyönä, 2007;Warren & McConnell, 2007). Here, we consider the role of animacy constraints and eventknowledge and how semantic relatedness interacts with both forms of semantic information.…”

Language and perception: Introduction to the Special Issue “Speakers and Listeners in the Visual World”

“…Another advantage of the VWP is its flexibility: it can be used to address questions at levels of language processing from the phonological to the discursive. The VWP has been used to study, among others, phonological effects on word recognition (Allopenna et al 1998), facilitative effects of selectional restrictions (Altmann and Kamide 1999), the mental representation of scenes (Altmann and Kamide 2009), and the role of event knowledge in predictive processing (Milburn et al 2016). Furthermore, the VWP is easy for participants to complete: participants are often required only to ''look and listen'' or to follow simple instructions, making it ideal for use in populations, such as older adults (Hayes et al 2016), people with aphasia (Mack et al 2013), young children (Borovsky et al 2012) or deficit populations (Norbury 2017;Vulchanova et al 2019).…”

“…To illustrate this, consider studies examining how scene information and event knowledge drive (predictive) eye movements. In a VWP study using naturalistic scenes, Milburn et al (2016) examined eye movements to targets driven both by information contained in the auditory verb stimulus and world knowledge conveyed by the scene depicted in the visual display. Participants viewed scenes and listened to sentences containing either constraining or unconstraining verbs.…”