A diverse range of factors affect the nature of neural representations underlying short-term memory

Abstract

Sequential and persistent activity models are two prominent models of short-term memory in neural circuits. In persistent activity models, memories are represented in persistent or nearly persistent activity patterns across a population of neurons, whereas in sequential models, memories are represented dynamically by a sequential pattern of activity across the population. Experimental evidence for both types of model in the brain has been reported previously. However, it has been unclear under what conditions these two qualitatively different types of solutions emerge in neural circuits. Here, we address this question by training recurrent neural networks on several short-term memory tasks under a wide range of circuit and task manipulations. We show that sequential and nearly persistent solutions are both part of a spectrum that emerges naturally in trained networks under different conditions. Fixed delay durations, tasks with higher temporal complexity, strong network coupling, motion-related dynamic inputs and prior training in a different task favor more sequential solutions, whereas variable delay durations, tasks with low temporal complexity, weak network coupling and symmetric Hebbian short-term synaptic plasticity favor more persistent solutions. Our results help clarify some seemingly contradictory experimental results on the existence of sequential vs. persistent activity based memory mechanisms in the brain..

Dynamic compression and expansion in a classifying recurrent network

“…We employ the most basic and widely used training rule for neural networks: stochastic gradient descent with backpropagation (through time). While not biologically plausible in its simplest form, the characteristics of networks trained by this algorithm can still resemble dynamics in the brain [38,53,4,41,5].…”

Interplay between persistent activity and activity-silent dynamics in prefrontal cortex during working memory

“…We proposed a computational model that can parsimoniously explain our data using short-term facilitation in the synapses of a recurrent network. Short-term plasticity has also been used in previous computational models of interacting activity-based and activity-silent dynamics 11,12,15 and of serial biases 22,27 . Beyond previous modeling efforts, we explored the mechanistic requirements of code reactivations prior to a new trial, and we derived predictions whose validation conferred plausibility to the model.…”

“…However, recent studies have argued that memories may be maintained without persistent firing rate tuning during memory periods 10 . This "activity-silent" memory can be mediated by short-term synaptic plasticity 11,12 , and possibly also by other activity-dependent intrinsic mechanisms with a long time constant [13][14][15] that could allow reactivation of memories from a latent storage. This computational proposal has received support in neuroimaging studies: in some working memory tasks, even if memory performance is good, stimulus information cannot be retrieved from neural recordings during delay, but later robustly reappears 16 during comparison or response periods (but see ref.…”

ORGaNICs: A Canonical Neural Circuit Computation

“…The literature on recurrent neural networks includes, of course, various complementary approaches that have each achieved some of the same goals (83)(84)(85)(86)(87)(88)(89)(90)(91)(92)(93)(94)(95). But none of the previously published models offer the same breadth of application of ORGaNICs.…”