12brain predictivity, with each model's untrained score predictive of its trained score. These results support the hypothesis that a 13 drive to predict future inputs may shape human language processing, and perhaps the way knowledge of language is learned 14 and organized in the brain. In addition, the finding of strong correspondences between ANNs and human representations opens 15 the door to using the growing suite of tools for neural network interpretation to test hypotheses about the human mind. 16 computational neuroscience, language comprehension, fMRI, ECoG, human brain recordings, natural language processing, artificial neural networks, deep learning Specific models accurately predict human brain activity. We found ( Fig. 2a-b) that specific models predict Pereira2018 and 114Fedorenko2016 datasets with up to 100% predictivity (see Fig. S2 for generalization to another metric) relative to the noise 115 ceiling (Methods-7, Fig. S1). The Blank2014 dataset is also reliably predicted, but with lower predictivity. Models vary 116 substantially in their ability to predict neural data. Generally, embedding models such as GloVe do not perform well on any 117 dataset. In contrast, recurrent networks such as skip-thoughts, as well as transformers such as BERT, predict large portions 118 of the data. The model that predicts the human data best across datasets is GPT2-xl, which predicts Pereira2018 and 119Fedorenko2016 at close to 100% and is among the highest-performing models on Blank2014 with 32% predictivity. These 120 scores are higher in the language network than other parts of the brain (SI-4). 121Model scores are consistent across experiments/datasets. To test the generality of the model representations, we examined the 122 consistency of model scores across datasets. Indeed, if a model does well on one dataset, it tends to also do well on other 123 datasets ( Fig. 2c), ruling out the possibility that we are picking up on spurious, dataset-idiosyncratic predictivity, and 124suggesting that the models' internal representations are general enough to capture brain responses to diverse linguistic 125 materials presented visually or auditorily, and across three independent sets of participants. Specifically, model scores 126across the two experiments in Pereira2018 (overlapping sets of participants) correlate at r=.94 (Pearson here and 127 elsewhere, p<<.00001), scores from Pereira2018 and Fedorenko2016 correlate at r=.50 (p<.001), and from Pereira2018 and 128Blank2014 at r=.63 (p<.0001). 129 130Next-word-prediction task performance predicts neural scores. In vision, ANNs that perform better on the specific task of 131 visual classification also tend to better predict responses in the primate ventral stream (Schrimpf et al., 2018; Yamins et al., 132 2014). Building on work that has established a core role for predictive processing in language (Hale, 2001; Levy, 2008a; 133 Smith & Levy, 2013) and recent findings that ANNs that perform well on a next-word prediction task (a normative task 134 known as 'languag...

The brain uses attention and expectation as flexible devices for optimizing behavioral responses associated with expected but unpredictably timed events. The neural bases of attention and expectation are thought to engage higher cognitive loci; however, their influence at the level of primary visual cortex (V1) remains unknown. Here, we asked whether single-neuron responses in monkey V1 were influenced by an attention task of unpredictable duration. Monkeys covertly attended to a spot that remained unchanged for a fixed period and then abruptly disappeared at variable times, prompting a lever release for reward. We show that monkeys responded progressively faster and performed better as the trial duration increased. Neural responses also followed monkey's task engagement-there was an early, but short duration, response facilitation, followed by a late but sustained increase during the time monkeys expected the attention spot to disappear. This late attentional modulation was significantly and negatively correlated with the reaction time and was well explained by a modified hazard function. Such bimodal, time-dependent changes were, however, absent in a task that did not require explicit attentional engagement. Thus, V1 neurons carry reliable signals of attention and temporal expectation that correlate with predictable influences on monkeys' behavioral responses.

No abstract

No abstract

No abstract