Spatial distance effects on incremental semantic interpretation of abstract sentences: Evidence from eye tracking

Abstract

a b s t r a c tA large body of evidence has shown that visual context information can rapidly modulate language comprehension for concrete sentences and when it is mediated by a referential or a lexical-semantic link. What has not yet been examined is whether visual context can also modulate comprehension of abstract sentences incrementally when it is neither referenced by, nor lexically associated with, the sentence. Three eye-tracking reading experiments examined the effects of spatial distance between words (Experiment 1) and objects (Experiment 2 and 3) on participants' reading times for sentences that convey similarity or difference between two abstract nouns (e.g., 'Peace and war are certainly different...'). Before reading the sentence, participants inspected a visual context with two playing cards that moved either far apart or close together. In Experiment 1, the cards turned and showed the first two nouns of the sentence (e.g., 'peace', 'war'). In Experiments 2 and 3, they turned but remained blank. Participants' reading times at the adjective (Experiment 1: first-pass reading time; Experiment 2: total times) and at the second noun phrase (Experiment 3: first-pass times) were faster for sentences that expressed similarity when the preceding words/objects were close together (vs. far apart) and for sentences that expressed dissimilarity when the preceding words/objects were far apart (vs. close together). Thus, spatial distance between words or entirely unrelated objects can rapidly and incrementally modulate the semantic interpretation of abstract sentences.

Prediction differs at sentence and discourse level: An event-related potential study

“…Recently, based on a large-scale (N = 334) study, Nieuwland and colleagues (2019) found that part of the late positivity effect of predictability may be explained in terms of semantic similarity. In addition, there is the possibility that semantic similarity effects are more likely to occur for words that appear close together (here, sentence context, album-pictures) than for distant relationships (discourse context, given that memory trace decreases with longer distance) (Guerra & Knoeferle, 2014). Therefore, we should not exclude the possibility that the positivity effect of predictability that we observed in the sentence condition is at least in part due to a semantic similarity effect.…”

Predicting (variability of) context effects in language comprehension

“…However, one might argue that cues in context differ at least sometimes in their appearance, timing, and relation to language and that we must derive a principled account of what languageworld relation elicits what effect on language comprehension (its time course and representations). To the extent that world-language relations govern context effects, the prediction from these findings is that verb-mediated action effects should influence comprehension and recall of the interpretation content more than cues that are not referentially tied into the interpretation [e.g., speaker gaze, relations between the queen was in agnoy and a depiction of the queen, emotional facial expressions cued by adjectives such as happy, distance between objects related to semantic similarity (Guerra and Knoeferle 2014)]. add evidence to the argument of distinct language-world relations modulating context effects: As participants in their studies listened to sentences such as Toby arrested the thief., they inspected both a picture of a thief and of a policeman to a similar extent and more than distractor objects during arrested the.…”

Extending Situated Language Comprehension (Accounts) with Speaker and Comprehender Characteristics: Toward Socially Situated Interpretation

“…Note that we are not claiming that these accounts, models or theories could not be adapted to include social context. We have selected one account (see Guerra and Knoeferle, 2014 , 2017 ; Knoeferle et al, 2014 on how the CIA can accommodate context effects in written language comprehension; see Knoeferle and Crocker, 2006 , 2007 for context effects in spoken language comprehension) to illustrate how enrichment with socially relevant information might be achieved.…”

“…The outcome of this reconciliation is tracked (match vs. mismatch) via indices to ant and int, and a truth value tracks the veracity of manual responses. Regarding int i′ , reconciliation involves coindexing representations of (minimally) nouns/verbs with those of depicted objects/actions (see Guerra and Knoeferle, 2014 , 2017 on non-referential co-indexing). Additionally, revision of the sentence interpretation can be informed by the scene.…”

“…The present article focuses on extending extant approaches to real-time situated language comprehension with effects of the socially interpreted context by explicitly integrating comprehender characteristics and representations of the speaker (of which more below, see (1) to (3)). Note that we use the term ‘comprehender’ instead of ‘listener’ to capture effects of the language user’s characteristics on both spoken and written language comprehension (see Guerra and Knoeferle, 2014 , 2017 ; Knoeferle et al, 2014 on how to accommodate visual context effects in written language comprehension; see Knoeferle and Crocker, 2006 , 2007 for accommodating visual context effects in spoken language comprehension). In integrating effects of the socially interpreted context on real-time language processing into a theoretical account, we must make several decisions.…”