Prediction (or not) during language processing. A commentary on Nieuwland et al. (2017) and DeLong et al. (2005)

Abstract

We would like to thank both DeLong and colleagues and Nieuwland and colleagues for sharing additional information about their studies with us, and for answering our many questions. We would also like to express that the points we raise in this manuscripts have not undergone peer review and should thus not be used to interfere with the publication of Nieuwland et al. (2017), which is still under review.. CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/143750 doi: bioRxiv preprint first posted online -2- (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/143750 doi: bioRxiv preprint first posted online -3- AbstractThe extent to which language processing involves prediction of upcoming inputs remains a question of ongoing debate. One important data point comes from DeLong et al. (2005) who reported that an N400-like event-related potential correlated with a probabilistic index of upcoming input. This result is often cited as evidence for gradient probabilistic prediction of form and/or semantics, prior to the bottom-up input becoming available. However, a recent multi-lab study reports a failure to find these effects . We review the evidence from both studies, including differences in the design and analysis approach between them. Building on over a decade of research on prediction since DeLong et al. (2005)'s original study, we also begin to spell out the computational nature of predictive processes that one might expect to correlate with ERPs that are evoked by a functional element whose form is dependent on an upcoming predicted word. For paradigms with this type of design, we propose an index of anticipatory processing, Bayesian surprise, and apply it to the updating of semantic predictions. We motivate this index both theoretically and empirically. We show that, for studies of the type discussed here, Bayesian surprise can be closely approximated by another, more easily estimated information theoretic index, the surprisal (or Shannon information) of the input. We re-analyze the data from Nieuwland and colleagues using surprisal rather than raw probabilities as an index of prediction. We find that surprisal is gradiently correlated with the amplitude of the N400, even in the data shared by Nieuwland and colleagues. Taken together, our review suggests that the evidence from both studies is compatible with anticipatory semantic processing. We do, however, emphasize the need for future studies to further clarify the nature and degree of form prediction, as well as its neural signatures, during language comprehension. Keywords: prediction; surprisal; Bayesian surprise; event-related potentials; hierarchical predictive processes; N400; N250 . CC-BY-NC-ND 4.0 Inter...

Similar time courses for word form and meaning preactivation during sentence comprehension

“…Ito, Martin, and Nieuwland likewise obtained a marginally significant ( p = 0.06) prediction N400 effect at a/an articles for native English‐speaking participants (see discussion in DeLong, Urbach, & Kutas, ). An exception to these findings is a controversial multilab experiment by Nieuwland et al (), which reported a failed replication of the DeLong et al study (see Yan, Kuperberg, & Jaeger, , and a blog post by Shravan Vasishth, : https://vasishth-statistics.blogspot.com/2017/04/a-comment-on-delong-et-al-2005-nine.html for elaboration).…”

“…Ito, Martin, and Nieuwland likewise obtained a marginally significant (p = 0.06) prediction N400 effect at a/an articles for native English-speaking participants (see discussion in DeLong, Urbach, & Kutas, 2017). An exception to these findings is a controversial multilab experiment by Nieuwland et al (2018), which reported a failed replication of the DeLong et al study (see Yan, Kuperberg, & Jaeger, 2017, Critically, others, using the same basic experimental paradigm as DeLong et al (2005), have taken their data as support for morphosyntactic prediction during sentence comprehension. These studies relied on gender-marked languages such as Spanish (Foucart, Martin, Moreno, & Costa, 2014;Wicha, Bates, et al, 2003;Wicha, Moreno, & Kutas, 2004) and Dutch (Van Berkum et al, 2005;Otten & Van Berkum, 2008) to show that readers and listeners preactivate specific upcoming nouns, as evidenced by amplitude modulations of ERPs to prenominal grammatical-gender-marked articles or adjectives.…”

Gricean Expectations in Online Sentence Comprehension: An ERP Study on the Processing of Scalar Inferences

“…Despite its explanatory success, the idea that predictive expectations explain online processing is not uncontroversial, and many crucial details remain unsettled. For instance, there is currently some vivid debate concerning the specific contextual cues that may trigger predictions, and about how specific and generalizable such predictions are (DeLong, Urbach, & Kutas, ; Ito, Martin, & Nieuwland, ; Yan, Kuperberg, & Jaeger, ). Moreover, it is astonishing that the vast majority of studies to date focuses on syntactic or lexical–semantic processing with much less emphasis on genuine pragmatic aspects (but see Werning & Cosentino, ).…”

Interplay between metrical and semantic processing in French: an N400 study

“…Because, while the N400 resulting from semantic incongruities is typically maximal in the centro-parietal region of the brain (Brown & Hagoort, 1993; Kutas & Federmeier, 2011), violations in metrical/phonological expectancies more commonly result in a N400 that is more frontally located (e.g. Böcker et al, 1999; DeLong et al, 2005; Lau et al, 2008; Steinhauer & Connolly, 2008; Rothermich et al, 2010, 2012; Yan et al, 2017), we selected fronto-central and centro-parietal electrodes (Fpz, FCz, Fz, AFz, Fp1, Fp2, FC1, FC2, F1, F2, AF3, AF4, Cz, P1, P2, C3, C4, Pz, P3, P4, CP1, CP2). Furthermore, because the phonological/metrical N400 has been reported to precede the semantic N400 temporally (e.g.…”

“… 1 Note that while the post-lexical integration theory may reject anticipatory processes and consider the N400 to index exclusively post-lexical processes initiated upon perceiving the target word, it not necessarily needs to; one can easily imagine integration processes to also benefit from successful (semantic) anticipation based on prior contextual information (as is pointed out by Yan et al 2017, see also Kuperberg & Jaeger 2016 and Nieuwland et al 2018). …”