A deep learning framework for neuroscience

Richards, Blake A.; Lillicrap, Timothy P.; Beaudoin, Philippe; Bengio, Yoshua; Bogacz, Rafał; Christensen, Amelia J.; Clopath, Claudia; Costa, Rui Ponte; Berker, Archy de; Ganguli, Surya; Gillon, Colleen J; Hafner, Danijar; Kepecs, Ádám; Kriegeskorte, Nikolaus; Latham, Peter E.; Lindsay, Grace W.; Miller, Kenneth D.; Naud, Richard; Pack, Christopher C.; Poirazi, Panayiota; Roelfsema, Pieter; Sacramento, João; Saxe, Andrew M.; Scellier, Benjamin; Schapiro, Anna C.; Senn, Walter; Wayne, Greg; Yamins, Daniel; Zenke, Friedemann; Zylberberg, Joel; Thérien, Denis; Kording, Konrad

doi:10.1038/s41593-019-0520-2

Cited by 757 publications

(701 citation statements)

References 74 publications

Supporting

Mentioning

624

Contrasting

Order By: Relevance

“…While we here present multiple approaches that can increase consistency (cocktail-blank, dropout, and the choice of distance measure), significant differences remain. For computational neuroscience to take full advantage of the deep learning framework (Cichy and Kaiser, 2019;Kietzmann et al, 2019a;Kriegeskorte and Douglas, 2018;Richards et al, 2019), we therefore suggest that DNNs should be treated similarly to experimental participants, as analyses should be based on groups of network instances. Representational consistency as defined here will give researchers a way to estimate the expected network variability for a given training scenario, and thereby enable them to better estimate how many networks are required to ensure that the insights drawn from them will generalize.…”

Section: Discussionmentioning

confidence: 99%

Individual differences among deep neural network models

Mehrer

Spoerer

Kriegeskorte

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modelling framework for neural computations in the primate brain. However, each DNN instance, just like each individual brain, has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using representational similarity analysis, we demonstrate that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations, despite achieving indistinguishable network-level classification performance. We locate the origins of the effects in an underconstrained alignment of category exemplars, rather than a misalignment of category centroids. Furthermore, while network regularization can increase the consistency of learned representations, considerable differences remain. These results suggest that computational neuroscientists working with DNNs should base their inferences on multiple networks instances instead of single off-the-shelf networks. Fig 1 | Characterizing network internal representations via representational similarity analysis and representational consistency. (A) Our comparisons of network internal representations were based on their multivariate activation patterns, extracted from each layer of each network instance as it responded to each of 1000 test images. (B) These high-dimensional activation vectors were then used to perform a representational similarity analysis (RSA). The fundamental building blocks of RSA are representational dissimilarity matrices (RDMs), which store all pairwise distances between the network's responses to the set of test stimuli. Each test image elicits a multivariate population response in each of the network's layers, which corresponds to a point in the respective high-dimensional activation space. The geometry of these points, captured in the RDM, provides insight into the nature of the representation, as it indicates which stimuli are grouped together, and which are separated. (C) To compare pairs of network instances, we compute their representational consistency, defined as the shared variance between network RDMs.

show abstract

Section: Discussionmentioning

confidence: 99%

Individual differences among deep neural network models

Mehrer

Spoerer

Kriegeskorte

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These results suggest that selection of the task goal is vastly more important for the resulting solutions than the particular architecture. Future work should put heavy emphasis on testing the differential effects of task goal, network architecture, and optimization procedure (Richards et al, 2019) .…”

Section: Discussionmentioning

confidence: 99%

A modular neural network model of grasp movement generation

Michaels

Schaffelhofer

Agudelo-Toro

et al. 2019

Preprint

View full text Add to dashboard Cite

One of the primary ways we interact with the world is using our hands. In macaques, the circuit spanning the anterior intraparietal area, the hand area of the ventral premotor cortex, and the primary motor cortex is necessary for transforming visual information into grasping movements. We hypothesized that a recurrent neural network mimicking the multi-area structure of the anatomical circuit and using visual features to generate the required muscle dynamics to grasp objects would explain the neural and computational basis of the grasping circuit. Modular networks with object feature input and sparse inter-module connectivity outperformed other models at explaining neural data and the inter-area relationships present in the biological circuit, despite the absence of neural data during network training. Network dynamics were governed by simple rules, and targeted lesioning of modules produced deficits similar to those observed in lesion studies, providing a potential explanation for how grasping movements are generated.

show abstract

“…Furthermore, since this long timescale 16 coincidence between two inputs is only permissive for BTSP, but does not determine the 17 sign of the plasticity, it cannot be classified as correlative in the same sense as classical 18 Hebbian or even anti-Hebbian forms of plasticity. As suggested by our network model, if 19 plateau potentials are generated by mismatch between a target instructive input and the 20 output of the local circuit, as reflected by dendritically-targeted feedback inhibition, 21 bidirectional BTSP can implement objective-based learning (48,49). In addition to 22 providing insight into the fundamental mechanisms of spatial memory formation in the 23 hippocampus, these findings suggest new directions for general theories of biological 1 learning and the development of artificial learning systems (44,50).…”

Section: D)mentioning

confidence: 99%

Bidirectional synaptic plasticity rapidly modifies hippocampal representations

Milstein

Bittner

et al. 2020

Preprint

View full text Add to dashboard Cite

15According to standard models of synaptic plasticity, correlated activity between 16 connected neurons drives changes in synaptic strengths to store associative 17 memories. Here we tested this hypothesis in vivo by manipulating the activity of 18 hippocampal place cells and measuring the resulting changes in spatial selectivity. 19 We found that the spatial tuning of place cells was rapidly reshaped via 20 bidirectional synaptic plasticity. To account for the magnitude and direction of 21 plasticity, we evaluated two models -a standard model that depended on 22 synchronous pre-and post-synaptic activity, and an alternative model that 23 1 depended instead on whether active synaptic inputs had previously been 1 potentiated. While both models accounted equally well for the data, they predicted 2 opposite outcomes of a perturbation experiment, which ruled out the standard 3 correlation-dependent model. Finally, network modeling suggested that this form 4 of bidirectional synaptic plasticity enables population activity, rather than pairwise 5 neuronal correlations, to drive plasticity in response to changes in the 6 environment. 7 8 Main Text 9 Activity-dependent changes in synaptic strength can flexibly alter the selectivity of 10 neuronal firing for particular features of the environment, providing a cellular substrate for 11 learning and memory. Various forms of Hebbian synaptic plasticity have been considered 12 for decades to be the main or even only synaptic plasticity mechanisms present within 13 most brain regions of a number of species. The core feature of such plasticity 14mechanisms is that they are autonomously driven by the repeated presence of correlated 15 presynaptic and postsynaptic activity that leads to either increases or decreases in 16 synaptic strength depending on the exact temporal coincidence (1-4). 17 The hippocampus plays an important role in many forms of learning and memory, 18 and the spatial firing rates of hippocampal place cells have been shown to change with 19 alterations in environmental context or the locations of salient features, like reward (5-20 11). Furthermore in CA1 neurons, place cell activity can emerge in a single trial following 21 a dendritic calcium spike (also called a plateau potential) (12-14). The form of synaptic 22 plasticity responsible for this rapid change in selectivity, termed behavioral timescale 23 2 synaptic plasticity (BTSP), modifies synaptic inputs active within a multi-second time 1 window around the plateau potential. That BTSP strengthens many synaptic inputs whose 2 activation did not cause or even coincide with postsynaptic activity suggests that it might 3 be a fundamentally different form of plasticity than classical correlation-driven Hebbian 4 plasticity (1-3). Such a plasticity rule could enable representation learning in cortical brain 5 regions like the hippocampus to be guided by delayed behavioral outcomes, rather than 6 by short timescale associations of neuronal input and output. However, it was not clear 7 from previous experiments...

show abstract

A deep learning framework for neuroscience

Cited by 757 publications

References 74 publications

Individual differences among deep neural network models

Individual differences among deep neural network models

A modular neural network model of grasp movement generation

Bidirectional synaptic plasticity rapidly modifies hippocampal representations

Contact Info

Product

Resources

About