Lexical surprisal shapes the time course of syntactic structure building

Abstract

When we understand language, we recognize words and combine them into sentences. How do we do this? In this paper, we explore the hypothesis that listeners use probabilistic information about words to build syntactic structure. Recent work has shown that lexical probability and syntactic structure both modulate the delta-band (0-4 Hz) neural signal. Here, we investigated whether the neural encoding of syntactic structure changes as a function of the distributional properties of a word. To this end, we analyzed MEG data of 24 native speakers of Dutch who listened to three fairytales with a total duration of 49 minutes. Using temporal response functions and a cumulative model-comparison approach, we evaluated the contributions of syntactic and distributional features to the variance in the delta-band neural signal. This revealed that lexical surprisal values (a distributional feature), as well as bottom-up node counts (a syntactic feature) positively contributed to the model of the delta-band neural signal. Subsequently, we compared responses to the syntactic feature between words with high- and low surprisal values. This revealed a delay in the response to the syntactic feature as a consequence of the surprisal value of the word: high surprisal values were associated with a delayed response to the syntactic feature by 150 to 190 milliseconds. The delay was not affected by word duration, and did not have a lexical origin. These findings suggest that the brain uses probabilistic information to infer syntactic structure, and highlight an importance for the role of time in this process.

Neural dynamics express syntax in the time domain during natural story listening

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain’s response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024).…”

“…These effects peaked twice within the first 500 ms after word onset and encompassed mostly inferior and middle frontal, and superior and middle temporal areas in the left hemisphere. The predictiveness of top-down node counts is somewhat at odds with previous studies that have looked at different parsing models in naturalistic comprehension, which either find that top-down methods are less predictive of brain activity (Giglio et al, 2024; Nelson et al, 2017; Slaats et al, 2024) or that they do not differ from other parsing methods (Brennan et al, 2016). What could account for the strong top-down effects?…”

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain's response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024). It thus appears to be the case that predictability modulates integratory structure building, but the story is not as straightforward as sketched in the main manuscript, at least not when a general measure of predictability like surprisal is used.…”

“…Another complicating factor is that the surprisal split is confounded not only with word duration, but also with word frequency (because surprisal is correlated with frequency, see Figure S2.1) and to a lesser extent also with entropy and the position of a word in a sentence (i.e., words in early-sentence positions are generally more surprising). All of these correlations might affect the pattern we see here, which underscores the importance of incorporating interactive effects in encoding models of language-related brain activity (see Slaats et al, 2024 for an exploration).…”

“…A possible implication of these findings is that parsing strategies can be flexibly adapted to the specific properties of the current linguistic input (e.g., grammatical properties, sentence complexity, reliability of predictive cues), such that people are less likely to engage in predictive structure building if the input contains ungrammatical sentences that make predicting ineffective (Brothers et al, 2019; Kuperberg & Jaeger, 2016). As an exploratory analysis of the potential trade-off between predictive and integratory structure building, we investigated whether predictability (quantified through surprisal) modulates the demands on bottom-up structure building (see the Supplementary Information S3; see also Slaats et al, 2024). How exactly the demands on predictive and integratory structure building dynamically change over the course of sentence processing should be investigated in future research.…”

Neural dynamics express syntax in the time domain during natural story listening

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain’s response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024).…”

“…These effects peaked twice within the first 500 ms after word onset and encompassed mostly inferior and middle frontal, and superior and middle temporal areas in the left hemisphere. The predictiveness of top-down node counts is somewhat at odds with previous studies that have looked at different parsing models in naturalistic comprehension, which either find that top-down methods are less predictive of brain activity (Giglio et al, 2024; Nelson et al, 2017; Slaats et al, 2024) or that they do not differ from other parsing methods (Brennan et al, 2016). What could account for the strong top-down effects?…”

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain's response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024). It thus appears to be the case that predictability modulates integratory structure building, but the story is not as straightforward as sketched in the main manuscript, at least not when a general measure of predictability like surprisal is used.…”

“…Another complicating factor is that the surprisal split is confounded not only with word duration, but also with word frequency (because surprisal is correlated with frequency, see Figure S2.1) and to a lesser extent also with entropy and the position of a word in a sentence (i.e., words in early-sentence positions are generally more surprising). All of these correlations might affect the pattern we see here, which underscores the importance of incorporating interactive effects in encoding models of language-related brain activity (see Slaats et al, 2024 for an exploration).…”

“…A possible implication of these findings is that parsing strategies can be flexibly adapted to the specific properties of the current linguistic input (e.g., grammatical properties, sentence complexity, reliability of predictive cues), such that people are less likely to engage in predictive structure building if the input contains ungrammatical sentences that make predicting ineffective (Brothers et al, 2019; Kuperberg & Jaeger, 2016). As an exploratory analysis of the potential trade-off between predictive and integratory structure building, we investigated whether predictability (quantified through surprisal) modulates the demands on bottom-up structure building (see the Supplementary Information S3; see also Slaats et al, 2024). How exactly the demands on predictive and integratory structure building dynamically change over the course of sentence processing should be investigated in future research.…”

Neural dynamics express syntax in the time domain during natural story listening

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain’s response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024).…”

“…These effects peaked twice within the first 500 ms after word onset and encompassed mostly inferior and middle frontal, and superior and middle temporal areas in the left hemisphere. The predictiveness of top-down node counts is somewhat at odds with previous studies that have looked at different parsing models in naturalistic comprehension, which either find that top-down methods are less predictive of brain activity (Giglio et al, 2024; Nelson et al, 2017; Slaats et al, 2024) or that they do not differ from other parsing methods (Brennan et al, 2016). What could account for the strong top-down effects?…”

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain's response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024). It thus appears to be the case that predictability modulates integratory structure building, but the story is not as straightforward as sketched in the main manuscript, at least not when a general measure of predictability like surprisal is used.…”

“…Another complicating factor is that the surprisal split is confounded not only with word duration, but also with word frequency (because surprisal is correlated with frequency, see Figure S2.1) and to a lesser extent also with entropy and the position of a word in a sentence (i.e., words in early-sentence positions are generally more surprising). All of these correlations might affect the pattern we see here, which underscores the importance of incorporating interactive effects in encoding models of language-related brain activity (see Slaats et al, 2024 for an exploration).…”

“…A possible implication of these findings is that parsing strategies can be flexibly adapted to the specific properties of the current linguistic input (e.g., grammatical properties, sentence complexity, reliability of predictive cues), such that people are less likely to engage in predictive structure building if the input contains ungrammatical sentences that make predicting ineffective (Brothers et al, 2019; Kuperberg & Jaeger, 2016). As an exploratory analysis of the potential trade-off between predictive and integratory structure building, we investigated whether predictability (quantified through surprisal) modulates the demands on bottom-up structure building (see the Supplementary Information S3; see also Slaats et al, 2024). How exactly the demands on predictive and integratory structure building dynamically change over the course of sentence processing should be investigated in future research.…”

Neural dynamics express syntax in the time domain during natural story listening

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain’s response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024).…”

“…These effects peaked twice within the first 500 ms after word onset and encompassed mostly inferior and middle frontal, and superior and middle temporal areas in the left hemisphere. The predictiveness of top-down node counts is somewhat at odds with previous studies that have looked at different parsing models in naturalistic comprehension, which either find that top-down methods are less predictive of brain activity (Giglio et al, 2024; Nelson et al, 2017; Slaats et al, 2024) or that they do not differ from other parsing methods (Brennan et al, 2016). What could account for the strong top-down effects?…”

“…Likewise, the temporal response function of bottom-up for low-surprisal words is more pronounced (Figures S3.1B and S3.1C). This suggests, contrary to our hypothesis, that the brain's response to the (syntactic properties of the) bottom-up input is stronger when the input can be predicted (for related results, and a more comprehensive exploration of the relation between syntactic and distributional information, see Slaats et al, 2024). It thus appears to be the case that predictability modulates integratory structure building, but the story is not as straightforward as sketched in the main manuscript, at least not when a general measure of predictability like surprisal is used.…”

“…Another complicating factor is that the surprisal split is confounded not only with word duration, but also with word frequency (because surprisal is correlated with frequency, see Figure S2.1) and to a lesser extent also with entropy and the position of a word in a sentence (i.e., words in early-sentence positions are generally more surprising). All of these correlations might affect the pattern we see here, which underscores the importance of incorporating interactive effects in encoding models of language-related brain activity (see Slaats et al, 2024 for an exploration).…”

“…A possible implication of these findings is that parsing strategies can be flexibly adapted to the specific properties of the current linguistic input (e.g., grammatical properties, sentence complexity, reliability of predictive cues), such that people are less likely to engage in predictive structure building if the input contains ungrammatical sentences that make predicting ineffective (Brothers et al, 2019; Kuperberg & Jaeger, 2016). As an exploratory analysis of the potential trade-off between predictive and integratory structure building, we investigated whether predictability (quantified through surprisal) modulates the demands on bottom-up structure building (see the Supplementary Information S3; see also Slaats et al, 2024). How exactly the demands on predictive and integratory structure building dynamically change over the course of sentence processing should be investigated in future research.…”