Recurrent Network Models of Sequence Generation and Memory

Rajan, Kanaka; Harvey, Christopher D.; Tank, David W.

doi:10.1016/j.neuron.2016.02.009

Cited by 345 publications

(466 citation statements)

References 74 publications

Supporting

Mentioning

432

Contrasting

Order By: Relevance

“…Nevertheless, even in its crudest form, the Potts network with its latching dynamics can be used to explore e.g., novel theories as to the evolutionary origin of complex cognition [39]. It establishes a quantitative framework to understand phase transitions [25], complementary to the perspective offered by other modelling approaches to sequence generation in cortical networks [40]. At the most abstract level, it can be considered an implementation of a fuzzy logic system [41,42], but with the critical advantage that its parameters can eventually be related to cortical parameters, as we begin to describe in a related study [32].…”

Section: Discussionmentioning

confidence: 99%

Life on the Edge: Latching Dynamics in a Potts Neural Network

Kang

Naim

Boboeva

et al. 2017

Entropy

View full text Add to dashboard Cite

Abstract:We study latching dynamics in the adaptive Potts model network, through numerical simulations with randomly and also weakly correlated patterns, and we focus on comparing its slowly and fast adapting regimes. A measure, Q, is used to quantify the quality of latching in the phase space spanned by the number of Potts states S, the number of connections per Potts unit C and the number of stored memory patterns p. We find narrow regions, or bands in phase space, where distinct pattern retrieval and duration of latching combine to yield the highest values of Q. The bands are confined by the storage capacity curve, for large p, and by the onset of finite latching, for low p. Inside the band, in the slowly adapting regime, we observe complex structured dynamics, with transitions at high crossover between correlated memory patterns; while away from the band latching, transitions lose complexity in different ways: below, they are clear-cut but last such few steps as to span a transition matrix between states with few asymmetrical entries and limited entropy; while above, they tend to become random, with large entropy and bi-directional transition frequencies, but indistinguishable from noise. Extrapolating from the simulations, the band appears to scale almost quadratically in the p-S plane, and sublinearly in p-C. In the fast adapting regime, the band scales similarly, and it can be made even wider and more robust, but transitions between anti-correlated patterns dominate latching dynamics. This suggest that slow and fast adaptation have to be integrated in a scenario for viable latching in a cortical system. The results for the slowly adapting regime, obtained with randomly correlated patterns, remain valid also for the case with correlated patterns, with just a simple shift in phase space.

show abstract

Section: Discussionmentioning

confidence: 99%

Life on the Edge: Latching Dynamics in a Potts Neural Network

Kang

Naim

Boboeva

et al. 2017

Entropy

View full text Add to dashboard Cite

show abstract

“…[31] [24], [3], [32], [28]), but the networks in these studies were all trained in a deterministic setting. An important recent development in deep learning has been the advent of the variational auto-encoder [19] [29], which combines a probabilistic framework with the power and ease of optimization of deep learning methods.…”

Section: Lfads Related Work In Machine Learning Literaturementioning

confidence: 99%

“…http://dx.doi.org/10.1101/152884 doi: [28]). It should be stressed that the vanilla RNN used as the data RNN here does not have the same functional form as the network generator used in the LFADS framework, which is a GRU (see section 1.6).…”

mentioning

confidence: 99%

Inferring single-trial neural population dynamics using sequential auto-encoders

Pandarinath

O’Shea

Collins

et al. 2017

Preprint

194

411

View full text Add to dashboard Cite

Neuroscience is experiencing a data revolution in which simultaneous recording of many hundreds or thousands of neurons is revealing structure in population activity that is not apparent from single-neuron responses. This structure is typically extracted from trial-averaged data. Single-trial analyses are challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. Here we introduce Latent Factor Analysis via Dynamical Systems (LFADS), a deep learning method to infer latent dynamics from single-trial neural spiking data. LFADS uses a nonlinear dynamical system (a recurrent neural network) to infer the dynamics underlying observed population activity and to extract 'de-noised' single-trial firing rates from neural spiking data. We apply LFADS to a variety of monkey and human motor cortical datasets, demonstrating its ability to predict observed behavioral variables with unprecedented accuracy, extract precise estimates of neural dynamics on single trials, infer perturbations to those dynamics that correlate with behavioral choices, and combine data from non-overlapping recording sessions (spanning months) to improve inference of underlying dynamics. In summary, LFADS leverages all observations of a neural population's activity to accurately model its dynamics on single trials, opening the door to a detailed understanding of the role of dynamics in performing computation and ultimately driving behavior.Increasing evidence suggests that in many brain areas, such as the motor and prefrontal cortices, the activity of large populations of neurons, termed the neural population state, is often well-described by low-dimensional dynamics [e.g. (Afshar et al. 2011; Harvey, Coen, and Tank 2012; Kaufman et al. 2014;Sadtler et al. 2014;Kobak et al. 2016a) ]. Recovering these dynamics on single trials is essential for illuminating the relationship between neural population activity and behavior, and for advancing therapeutic neurotechnologies such as closed-loop deep brain stimulation and brain-machine interfaces. However, recovering population dynamics All rights reserved. No reuse allowed without permission.(which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/152884 doi: bioRxiv preprint first posted online Jun. 20, 2017; on single trials is difficult due to trial-to-trial variability (e.g. in behavior or arousal) and fluctuations in the spiking of individual neurons. Even with dramatic increases in the numbers of neurons that can be simultaneously recorded using multichannel electrode arrays or optical imaging, accurately recovering population dynamics from single trials remains a significant challenge for data-analysis methods.Standard analyses sacrifice single-trial information for the sake of better estimates of trial-averaged neural states (Ahrens et al. 2012; Kobak et al. 2016b) . Techniques for extrac...

show abstract

“…In recent years, there has been renewed interest in modeling complex human behaviors such as memory and motor skills using neural networks (Sussillo et al 2015;Rajan, Harvey, and Tank 2016;Hennequin, Vogels, and Gerstner 2014;Carnevale et al 2015;Laje, Buonomano, and Buonomano 2013). However, training these networks to produce meaningful behavior has proven difficult.…”

Section: Discussionmentioning

confidence: 99%