Constructing neural network models from brain data reveals representational transformations linked to adaptive behavior

Ito, Takuya; Yang, Guangyu Robert; Laurent, Patryk A.; Schultz, Douglas H.; Cole, Jennifer

doi:10.1038/s41467-022-28323-7

Cited by 38 publications

(37 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Expanding dimensionality improves representation and classification, whereas reducing dimensionality improves generalizability to unfamiliar contexts [42, 43, 44, 45, 29]. Heterogenous levels of dimensionality support flexible behavior and compositionality of compact representations [46, 47, 48]. To further understand information processing, the model could be used to explain how the brain generates and updates compressed models of the environment with differing constraints of dimensionality, timescales, and controllability [49, 50, 39, 45], key elements of predictive coding [51, 52, 16].…”

Section: Discussionmentioning

confidence: 99%

Compression supports low-dimensional representations of behavior across neural circuits

Zhou

Kim

Pines

et al. 2022

Preprint

View full text Add to dashboard Cite

Dimensionality reduction, a form of compression, can simplify representations of information to increase efficiency and reveal general patterns. Yet, this simplification also forfeits information, thereby reducing representational capacity. Hence, the brain may benefit from generating both compressed and uncompressed activity, and may do so in a heterogeneous manner across diverse neural circuits that represent low-level (sensory) or high-level (cognitive) stimuli. However, precisely how compression and representational capacity differ across the cortex remains unknown. Here we predict different levels of compression across regional circuits by using random walks on networks to model activity flow and to formulate rate-distortion functions, which are the basis of lossy compression. Using a large sample of youth (n=1,040), we test predictions in two ways: by measuring the dimensionality of spontaneous activity from sensorimotor to association cortex, and by assessing the representational capacity for 24 behaviors in neural circuits and 20 cognitive variables in recurrent neural networks. Our network theory of compression predicts the dimensionality of activity (t=12.13, p<0.001) and the representational capacity of biological (r=0.53, p=0.016) and artificial (r=0.61, p<0.001) networks. The model suggests how a basic form of compression is an emergent property of activity flow between distributed circuits that communicate with the rest of the network.

show abstract

Section: Discussionmentioning

confidence: 99%

Compression supports low-dimensional representations of behavior across neural circuits

Zhou

Kim

Pines

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The following description is quoted with citation from a previous study that used the same dataset [23]. The dataset was publicly published under a CC0 license, and is publicly available (https://openneuro.org/ datasets/ds003701).…”

Section: A Supplementary Methodsmentioning

confidence: 99%

“…The following fMRI acquisition details are taken from a previous study that used the same data set [23].…”

Section: A4 Fmri Acquisitionmentioning

confidence: 99%

See 1 more Smart Citation

Compositional generalization through abstract representations in human and artificial neural networks

Ito¹,

Klinger²,

Schultz³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Humans have a remarkable ability to rapidly generalize to new tasks that is difficult to reproduce in artificial learning systems. Compositionality has been proposed as a key mechanism supporting generalization in humans, but evidence of its neural implementation and impact on behavior is still scarce. Here we study the computational properties associated with compositional generalization in both humans and artificial neural networks (ANNs) on a highly compositional task. First, we identified behavioral signatures of compositional generalization in humans, along with their neural correlates using whole-cortex functional magnetic resonance imaging (fMRI) data. Next, we designed pretraining paradigms aided by a procedure we term primitives pretraining to endow compositional task elements into ANNs. We found that ANNs with this prior knowledge had greater correspondence with human behavior and neural compositional signatures. Importantly, primitives pretraining induced abstract internal representations, excellent zero-shot generalization, and sample-efficient learning. Moreover, it gave rise to a hierarchy of abstract representations that matched human fMRI data, where sensory rule abstractions emerged in early sensory areas, and motor rule abstractions emerged in later motor areas. Our findings give empirical support to the role of compositional generalization in human behavior, implicate abstract representations as its neural implementation, and illustrate that these representations can be embedded into ANNs by designing simple and efficient pretraining procedures.

show abstract

“…In support of this prediction, studies have found that associative regions in frontal and parietal cortices are involved in executing a wide array of tasks ( Cole et al, 2013 ; Duncan, 2010 ). These task-flexible regions, also commonly referred to as brain hubs ( Gratton et al, 2018 ; van den Heuvel and Sporns, 2013 ), have diverging connectivity with multiple brain systems, and are thought to perform integrative functions that allow perceptual inputs to interact with contextual task representations for adaptive task control ( Bertolero et al, 2018 ; Bertolero et al, 2015 ; Ito et al, 2022 ; Nee, 2021 ). The behavioral significance of brain hubs is affirmed by lesion studies demonstrating that lesions to hub regions are associated with task impairments across multiple functional domains ( Hwang et al, 2021 ; Reber et al, 2021 ; Warren et al, 2014 ).…”

Section: Introductionmentioning

confidence: 99%

Thalamocortical contributions to cognitive task activity

Hwang

Shine

Cole

et al. 2022

eLife

View full text Add to dashboard Cite

Thalamocortical interaction is a ubiquitous functional motif in the mammalian brain. Previously (Hwang et al., 2021), we reported that lesions to network hubs in the human thalamus are associated with multi-domain behavioral impairments in language, memory, and executive functions. Here we show how task-evoked thalamic activity are organized to support these broad cognitive abilities. We analyzed functional MRI data from human subjects that performed 127 tasks encompassing a broad range of cognitive representations. We first investigated the spatial organization of task-evoked activity and found a basis set of activity patterns evoked to support processing needs of each task. Specifically, the anterior, medial, and posterior-medial thalamus exhibit hub-like activity profiles that are suggestive of broad functional participation. These thalamic task hubs overlapped with network hubs interlinking cortical systems. To further determine the cognitive relevance of thalamic activity and thalamocortical functional connectivity, we built a data-driven thalamocortical model to test whether thalamic activity can be used to predict cortical task activity. The thalamocortical model predicted task-specific cortical activity patterns, and outperformed comparison models built on cortical, hippocampal, and striatal regions. Simulated lesions to low-dimensional, multi-task thalamic hub regions impaired task activity prediction. This simulation result was further supported by profiles of neuropsychological impairments in human patients with focal thalamic lesions. In summary, our results suggest a general organizational principle of how the human thalamocortical system supports cognitive task activity.

show abstract

Constructing neural network models from brain data reveals representational transformations linked to adaptive behavior

Cited by 38 publications

References 64 publications

Compression supports low-dimensional representations of behavior across neural circuits

Compression supports low-dimensional representations of behavior across neural circuits

Compositional generalization through abstract representations in human and artificial neural networks

Thalamocortical contributions to cognitive task activity

Contact Info

Product

Resources

About