Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks

Mastrogiuseppe, Francesca; Ostojic, Srdjan

doi:10.1016/j.neuron.2018.07.003

Cited by 286 publications

(501 citation statements)

References 55 publications

(129 reference statements)

Supporting

Mentioning

459

Contrasting

Order By: Relevance

“…Asymmetric couplings have been previously used to generate specific temporal dynamics, though in the absence of noise. Examples include models of temporal sequences [27,28] or recurrent networks within the echo-state/reservoir computing framework [78][79][80][81]. All these models are fundamentally different from ours, as their low-rank structure is fixed and time-independent, hence their temporal dynamics entails no trial-to-trial variability.…”

Section: Correlated Variability In Sensory Vs Motor Processingmentioning

confidence: 99%

Metastable attractors explain the variable timing of stable behavioral action sequences

Recanatesi

Pereira

Murakami

et al. 2020

Preprint

View full text Add to dashboard Cite

Natural animal behavior displays rich lexical and temporal dynamics, even in a stable environment. This implies that behavioral variability arises from sources within the brain, but the origin and mechanics of these processes remain largely unknown. Here, we focus on the observation that the timing of self-initiated actions shows large variability even when they are executed in stable, well-learned sequences. Could this mix of reliability and stochasticity arise within the same circuit? We trained rats to perform a stereotyped sequence of self-initiated actions and recorded neural ensemble activity in secondary motor cortex (M2), which is known to reflect trial-by-trial action timing fluctuations. Using hidden Markov models we established a robust and accurate dictionary between ensemble activity patterns and actions. We then showed that metastable attractors, representing activity patterns with the requisite combination of reliable sequential structure and high transition timing variability, could be produced by reciprocally coupling a high dimensional recurrent network and a low dimensional feedforward one. Transitions between attractors were generated by correlated variability arising from the feedback loop between the two networks. This mechanism predicted a specific structure of low-dimensional noise correlations that were empirically verified in M2 ensemble dynamics. This work suggests a robust network motif as a novel mechanism to support critical aspects of animal behavior and establishes a framework for investigating its circuit origins via correlated variability.

show abstract

Section: Correlated Variability In Sensory Vs Motor Processingmentioning

confidence: 99%

Metastable attractors explain the variable timing of stable behavioral action sequences

Recanatesi

Pereira

Murakami

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…To investigate which factors influence learning within and outside of the neural manifold, we used a recurrent neural network trained with a machine learning algorithm. Although this method of modelling neural dynamics lacks biological details, using RNNs has been surprisingly useful to understand neural phenomena [Sussillo et al, 2015, Barak, 2017, Mastrogiuseppe and Ostojic, 2018, Michaels et al, 2019, Masse et al, 2019. Using this approach, we could identify feedback learning as a potential bottleneck differentiating between learning within versus outside the original manifold.…”

Section: Discussionmentioning

confidence: 99%

“…One potential reason is that most experimental designs are inherently low-dimensional, and therefore bias the observed neural activity [Gao et al, 2017]. Alternatively, it has been shown that low-dimensional dynamics can arise from structured connectivity within the network [Sussillo and Abbott, 2009, Laje and Buonomano, 2013, Hu et al, 2014, Hennequin et al, 2014, Aljadeff et al, 2016, Rivkind and Barak, 2017, Mastrogiuseppe and Ostojic, 2018, DePasquale et al, 2018, Recanatesi et al, 2019.…”

Section: Introductionmentioning

confidence: 99%

Neural manifold under plasticity in a goal driven learning behaviour

Feulner

Clopath

2020

Preprint

View full text Add to dashboard Cite

Neural activity is often low dimensional and dominated by only a few prominent neural covariation patterns. It has been hypothesised that these covariation patterns could form the building blocks used for fast and flexible motor control. Supporting this idea, recent experiments have shown that monkeys can learn to adapt their neural activity in motor cortex on a timescale of minutes, given that the change lies within the original low-dimensional subspace, also called neural manifold. However, the neural mechanism underlying this within-manifold adaptation remains unknown. Here, we show in a computational model that modification of recurrent weights, driven by a learned feedback signal, can account for the observed behavioural difference between within-and outside-manifold learning. Our findings give a new perspective, showing that recurrent weight changes do not necessarily lead to change in the neural manifold. On the contrary, successful learning is naturally constrained to a common subspace.

show abstract

“…where the u (i) and v (i) are N -dimensional vectors. Connectivity matrices of this form are called low-rank connectivities (Mastrogiuseppe and Ostojic, 2018;Hopfield, 1982). They belong to the broader class of non-normal matrices (Trefethen and Embree (2005); Murphy and Miller (2009); Goldman (2009); Hennequin et al (2012); see Methods) and can produce amplified transient dynamics in responses to specific stimuli.…”

Section: A Recurrent Network Model For Off Responses: a Dynamical Sysmentioning

confidence: 99%

“…This feature is preserved even when the fitting is performed on a smaller subset of units (subsampling 50% and 20% of the units). In A.-C. we used a recurrent connectivity J = J low−rank + gχ, given by the sum of a low-rank connectivity (with rank 30) and a Gaussian connectivity with mean zero and standard deviation g/ √ N (with g = 0.2), which acts as connectivity noise (Mastrogiuseppe and Ostojic, 2018;Bondanelli and Ostojic, 2018). 20 OFF responses were generated by setting the state before stimulus offset along the first 20 amplified initial conditions.…”

Section: Recurrent Modelmentioning

confidence: 99%

Network dynamics underlying OFF responses in the auditory cortex

Bondanelli

Deneux

Bathellier

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Across sensory systems, complex spatio-temporal patterns of neural activity arise following the onset (ON) and offset (OFF) of simple stimuli. While ON responses have been widely studied, the mechanisms generating OFF responses in cortical areas have so far not been fully elucidated. Recent studies have argued that OFF responses reflect strongly transient sensory coding at the population level, suggesting they may be generated by a collective, network mechanism. We examine here the hypothesis that OFF responses are single-cell signatures of network dynamics and propose a simple model that generates transient OFF responses through recurrent interactions. To test this hypothesis, we analyse the responses of large populations of neurons in auditory cortex recorded using two-photon calcium imaging in awake mice passively listening to auditory stimuli. Adopting a population approach, we find that the OFF responses evoked by individual stimuli define trajectories of activity that evolve in low-dimensional neural subspaces. A geometric analysis of the population OFF responses reveals that, across different stimuli, these neural subspaces lie on largely orthogonal dimensions that define low-dimensional transient coding channels. We show that a linear network model with recurrent interactions provides a simple mechanism for the generation of the strong OFF responses observed in auditory cortex, and accounts for the low-dimensionality of the transient channels and their global structure across stimuli. We finally compare the network model with a single-cell mechanism and show that the single-cell model, while explaining some prominent features of the data, does not account for the structure across stimuli captured by the network model.

show abstract

Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks

Cited by 286 publications

References 55 publications

Metastable attractors explain the variable timing of stable behavioral action sequences

Metastable attractors explain the variable timing of stable behavioral action sequences

Neural manifold under plasticity in a goal driven learning behaviour

Network dynamics underlying OFF responses in the auditory cortex

Contact Info

Product

Resources

About