Expectation-based syntactic comprehension

Abstract

This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple informationtheoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale (2001), in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.

Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading

“…An obvious question is why we did not observe surprisal effects in the eyetracking measures given past reports of surprisal effects on reading time (Boston et al, 2008;Demberg and Keller, 2008;Demberg et al, 2013;Levy, 2008). It is important to note, however, that these earlier studies investigated either overall surprisal (i.e., a combination of n-gram and lexicalized syntactic surprisal) or un-lexicalized part-of-speech (POS) surprisal, both of which are different from the surprisal measure we investigated.…”

“…In our study, we capitalized on this work to quantify syntactic difficulty using a metric called surprisal (Hale, 2001;Levy, 2008). Surprisal is an information-theoretic concept that reflects the expectedness of each word given its preceding context.…”

“…Surprisal is an information-theoretic concept that reflects the expectedness of each word given its preceding context. Surprisal is an increasingly influential concept in the language sciences because it reflects moment-by-moment processing operations related to the system's attempts to connect the current input to the left context, based on the comprehender's history of language use (Boston et al, 2008;Demberg and Keller, 2008;Hale, 2001;Levy, 2008). Unlike some other measures of text complexity, a surprisal value can be generated for every word of a text, making it methodologically feasible to relate syntactic difficulty to brain activity on a word-by-word basis during the reading of connected texts.…”

“…Higher surprisal values are associated with greater processing difficulty: Surprisal values reliably predict word-by-word reading times derived from eye movement measures (e.g., Boston et al, 2008;Demberg and Keller, 2008;Demberg et al, 2013;Levy, 2008), the N400 event-related potential (ERP) component (Frank et al, 2015), and fMRI activation (Hale et al, 2015;Willems et al, in press).…”

Data from eye-tracking corpora as evidence for theories of syntactic processing complexity

“…Intuitively, the surprisal of a word in a sentential context corresponds to the probability mass of the analyses that are not consistent with the new word. Surprisal requires a probabilistic notion of linguistic structure (utilizing transitional probabilities or probabilistic grammars), and has its theoretical foundation in information theory (Levy, 2008). It can be shown to capture a range of complexity effects, including the subject/object relative clause asymmetry, certain garden path effects, speed-up effects in verb-final contexts, and word order asymmetries in German (Hale, 2001;Levy, 2008).…”