Neurobehavioral Correlates of Surprisal in Language Comprehension: A Neurocomputational Model

Abstract

Expectation-based theories of language comprehension, in particular Surprisal Theory, go a long way in accounting for the behavioral correlates of word-by-word processing difficulty, such as reading times. An open question, however, is in which component(s) of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these electrophysiological correlates relate to behavioral processing indices. Here, we address this question by instantiating an explicit neurocomputational model of incremental, word-by-word language comprehension that produces estimates of the N400 and the P600—the two most salient ERP components for language processing—as well as estimates of “comprehension-centric” Surprisal for each word in a sentence. We derive model predictions for a recent experimental design that directly investigates “world-knowledge”-induced Surprisal. By relating these predictions to both empirical electrophysiological and behavioral results, we establish a close link between Surprisal, as indexed by reading times, and the P600 component of the ERP signal. The resultant model thus offers an integrated neurobehavioral account of processing difficulty in language comprehension.

A Framework for Distributional Formal Semantics

“…The complementary nature of these meaning representations is underlined by recent advances in the neurocognition of language, where evidence suggests that lexical retrieval (the mapping of words onto lexical semantics) and semantic integration (the integration of word meaning into the unfolding representation of propositional meaning) are two distinct processes involved in word-by-word sentence processing (see [44,45] for explicit neurocognitive models). More specifically, this neurocognitive perspective on comprehension suggests that there is no compositionality at the lexical level; that is, word forms in context are mapped into word meaning representations, which are subsequently integrated into a compositional phrasal-/utterance-level meaning representation.…”

Semantic Systematicity in Connectionist Language Production

“…For example, when the model processes "a boy plays soccer", it does not only recover the explicit literal propositional content, but it also constructs a more complete situation model, in which a boy is likely to be playing outside, on a field, with a ball, etc. In this way, it differs from other connectionist models of language processing, that typically employ simpler meaning representations, such as case-roles (e.g., [26][27][28][29]). Crucially, Frank et al [23]'s model generalizes to sentences and meaning representations that it has not seen during training, exhibiting different levels of semantic systematicity.…”

Retrieval (N400) and integration (P600) in expectation-based comprehension

“…This Surprisal measure is influenced by both linguistic experience, as well as knowledge about the world [ 5 ]. As Brouwer et al [ 48 ] point out, this view of the P600 as reflecting comprehension-centric Surprisal follows from the RI Theory. Just as syntactic models determine the likelihood of alternative analyses based on linguistic experience, the RI model recovers interpretations that reflect the distributional characteristics of the utterances it is exposed to [ 48 ].…”

“…In fact, the decomposition of language comprehension into retrieval and integration is made even more explicit in the computational instantiation of RI theory. In this model, retrieval is instantiated by the function which maps an incoming orthographic/acoustic word form onto a representation of word meaning , while taking the unfolding utterance context —the utterance meaning constructed prior to the current word—into account [ 48 ]. The output of this function serves as input to the function which serves to integrate the retrieved word meaning into the unfolding utterance context , to produce an updated utterance meaning .…”

“…The most recent instantiation of RI theory thus predicts the P600 component of the ERP signal, which indexes the amount of effort involved in updating the unfolding utterance meaning representation with the retrieved meaning of an incoming word, to be the locus that is specifically sensitive to expectancy/Surprisal effects [ 48 ] and insensitive to association effects. That is, integration effort is assumed to increase to the extent that the utterance meaning representation resulting from integrating this word meaning is semantically, pragmatically, or structurally unexpected, given the utterance meaning representation prior to integration.…”