Cortical processing of reference in language revealed by computational models

Li, Jixing; Wang, Shaonan; Luh, Wen‐Ming; Pylkkänen, Liina; Yang, Yujiu; Hale, John

doi:10.1101/2020.11.24.396598

Cited by 7 publications

(4 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, many recent studies have shown that neural network language models (NNLMs), which embody (some elements of) predictive coding theory, are much more effective at explaining brain activity elicited by natural language than earlier methods ( Anderson et al, 2021 ; Antonello et al, 2021 ; Caucheteux et al, 2021a ; Caucheteux & King, 2022 ; Goldstein et al, 2021 ; Jain & Huth, 2018 ; Jat et al, 2019 ; LeBel, Jain, & Huth, 2021 ; Li et al, 2021 ; Schrimpf et al, 2021 ; Tikochinski et al, 2021 ; Toneva et al, 2020 ). Some of these studies claim that the superiority of NNLMs over other methods is evidence for predictive coding theory in language ( Goldstein et al, 2021 ; Schrimpf et al, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

Antonello

Huth

2023

Neurobiology of Language

View full text Add to dashboard Cite

Many recent studies have shown that representations drawn from neural network language models are extremely effective at predicting brain responses to natural language. But why do these models work so well? One proposed explanation is that language models and brains are similar because they have the same objective: to predict upcoming words before they are perceived. This explanation is attractive because it lends support to the popular theory of predictive coding. We provide several analyses that cast doubt on this claim. First, we show that the ability to predict future words does not uniquely (or even best) explain why some representations are a better match to the brain than others. Second, we show that within a language model, representations that are best at predicting future words are strictly worse brain models than other representations. Finally, we argue in favor of an alternative explanation for the success of language models in neuroscience: these models are effective at predicting brain responses because they generally capture a wide variety of linguistic phenomena.

show abstract

Section: Introductionmentioning

confidence: 99%

Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

Antonello

Huth

2023

Neurobiology of Language

View full text Add to dashboard Cite

show abstract

“…We stress the importance of considering multiple languages when building and testing neurobiological models of language processing, assuming that the neural substrates and processes of language are shared among speakers of all languages. As shown in previous work examining coreference resolution using the English and Chinese subset of this corpus, the computational model that best explains the neural signature for pronoun processing is generalizable for both English and Chinese 18 . These data can be reused to address different research questions with a variety of analytical methods.…”

Section: Background and Summarymentioning

confidence: 53%

“…Although GLM or encoding models have been commonly applied to fMRI data using long naturalistic stimuli like audiobooks 8,9,11,18,[38][39][40] . There are no standardised approaches for analysing complex and high dimensional naturalistic fMRI data.…”

Section: Analysis Bottleneckmentioning

confidence: 99%

Le Petit Prince: A multilingual fMRI corpus using ecological stimuli

Bhattasali

Zhang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Neuroimaging using more ecologically valid stimuli such as audiobooks has advanced our understanding of natural language comprehension in the brain. However, prior naturalistic stimuli have typically been restricted to a single language, which limited generalizability beyond small typological domains. Here we present the "Le Petit Prince fMRI Corpus" (LPPC-fMRI), a multilingual resource for research in the cognitive neuroscience of speech and language during naturalistic listening (Open-Neuro: ds003643). 49 English speakers, 35 Chinese speakers and 28 French speakers listened to the same audiobook "The Little Prince" in their native language while multi-echo functional magnetic resonance imaging was acquired. We also provide time-aligned speech annotation and word-by-word predictors obtained using natural language processing tools. The resulting timeseries data are shown to be of high quality with good temporal signal-to-noise ratio and high inter-subject correlation. Data-driven functional analyses provide further evidence of data quality. This annotated, multilingual fMRI dataset facilitates future re-analysis that addresses cross-linguistic commonalities and differences in the neural substrate of language processing on multiple perceptual and linguistic levels.

show abstract

“…Research on language cognition also draws on the operating mechanisms of language-computation models to propose hypotheses. A representative example of this type of work is that of Li et al [109]. They used a cognitive model and a neural network model to study the neural mechanism of the brain when understanding pronouns.…”

Section: Language Cognition Experiments Using Language Computation Me...mentioning

confidence: 99%

Language Cognition and Language Computation -- Human and Machine Language Understanding

Shaonan¹,

Ding²,

Lin³

et al. 2023

Preprint

View full text Add to dashboard Cite

Language understanding is a key scientific issue in the fields of cognitive and computer science. However, the two disciplines differ substantially in the specific research questions. Cognitive science focuses on analyzing the specific mechanism of the brain and investigating the brain's response to language; few studies have examined the brain's language system as a whole. By contrast, computer scientists focus on the efficiency of practical applications when choosing research questions but may ignore the most essential laws of language. Given these differences, can a combination of the disciplines offer new insights for building intelligent language models and studying language cognitive mechanisms? In the following text, we first review the research questions, history, and methods of language understanding in cognitive and computer science, focusing on the current progress and challenges. We then compare and contrast the research of language understanding in cognitive and computer sciences. Finally, we review existing work that combines insights from language cognition and language computation and offer prospects for future development trends. * This paper is originally written in Chinese and published in SCIENTIA SINICA Informationis. Here we translate it into English with an extension of recent work.

show abstract

Cortical processing of reference in language revealed by computational models

Cited by 7 publications

References 89 publications

Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

Le Petit Prince: A multilingual fMRI corpus using ecological stimuli

Language Cognition and Language Computation -- Human and Machine Language Understanding

Contact Info

Product

Resources

About