A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence

Abstract

Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. We present the Natural Scenes Dataset (NSD), in which high-resolution fMRI responses to tens of thousands of richly annotated natural scenes are measured while participants perform a continuous recognition task. To optimize data quality, we develop and apply novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we use NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality, and breadth, NSD opens new avenues of inquiry in cognitive and computational neuroscience..

What has vision science taught us about functional MRI?

“…Attention has been placed on the reproducibility of psychology (Open Science Collaboration, 2015) and neuroimaging studies (Botvinik-Nezer et al, 2020;Poldrack et al, 2017). Human retinotopic maps are highly reproducible (Benson et al, 2018;Himmelberg et al, 2021;Lage-Castellanos et al, 2020;Lerma-Usabiaga et al, 2020;van Dijk et al, 2016) and large, publicly available datasets of fMRI responses in visual cortex, such as the HCP Retinotopy (Benson et al, 2018) and NSD datasets (Allen et al, 2022), are at the forefront of understanding brain function. This high level of reproducibility is due to the implementation of an explicit computational approach in characterising the fMRI signal.…”

Towards real-world neuroscience using mobile EEG and augmented reality

“…This is an important and fruitful approach, and will continue to be so. As cognitive neuroscience methods progress, a complementary approach has also become more feasible, namely, research using more naturalistic paradigms which look towards the functioning of the human mind unleashed from controlled experiments [1][2][3][4] . Moreover, experimentation beyond the lab is possible by building on the platform established by prior approaches, and by employing newer and emerging technologies to study the human brain-such as virtual reality, augmented reality and fully mobile approaches for neural recordings.…”

Normal Retinotopy in Primary Visual Cortex in a Congenital Complete Unilateral Lesion of Lateral Geniculate Nucleus in Human: A Case Study

“…Figure 3A shows that the both the LGN and the pulvinar respond well to stimuli centered in the eight visual field locations (three wedges located in the left, three located in the right hemifield, and two located at the cardinal meridians), suggesting that they represent both the ipsi-lateral and the contra-lateral visual field signal. Both ROIs were taken from a recently released atlas of the thalamic nuclei [24] and projected on the right hemisphere of IB. Finally, pRF maps revealed an organized retinotopic map in the activated thalamic region with an ipsilateral visual field preference (Figure 3B).…”

“…Pulvinar and LGN ROIs in the intact hemisphere were mapped from the MNI atlas created as a part of the Natural Scenes Dataset [24]. To do that, we used a non-linear warping technique implemented in ANTs [41].…”