Phase of firing does not reflect temporal order in sequence memory of humans and recurrent neural networks

Abstract

A prominent theory proposes that the temporal order of a sequence of items held in memory is reflected in ordered firing of neurons at different phases of theta oscillations (Lisman & Idiart, 1995). We probe this theory by directly measuring single neuron activity (1420 neurons) and local field potentials (LFP, 921 channels) in the medial temporal lobe of 16 epilepsy patients performing a working memory task for temporal order. We observe theta oscillations and preferential firing of single neurons at theta phase during memory maintenance. We find that - depending on memory performance - phase of firing is related to item position within a sequence. However, in contrast to the theory, phase order did not match item order. To investigate underlying mechanisms, we subsequently trained recurrent neural networks (RNNs) to perform an analogous task. Similar to recorded neural activity, we show that RNNs generate theta oscillations during memory maintenance. Importantly, model neurons exhibit theta phase-dependent firing related to item position, where phase of firing again did not match item order. Instead, we observed a mechanistic link between phase order, stimulus timing and oscillation frequency - a relationship we subsequently confirmed in our neural recordings. Taken together, in both biological and artificial neural networks we provide validating evidence for the role of phase-of-firing in memory processing while at the same time challenging a long-held theory about the functional role of spiking and oscillations in sequence memory.

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

“…S8. Phase coding has also been observed during memory of sequences of information [16], as well as for coding of position [10]. Linking these findings to our model requires further investigation.…”

“…The network consists of N units, with activation x ( t ) ∈ ℛ N , recurrently connected via a connectivity matrix J ∈ ℛ N ×N , and receiving external oscillatory input u ( t ) ∈ ℛ [7, 18, 39], as well as stimuli s ( t ) ∈ ℛ 2 , where τ represents the time constant of the units, tanh is an elementwise non-linearity, I ( osc ) ∈ ℛ N , I ( s ) ∈ ℛ N× 2 represent the input weights, and ξ ( t ) ∈ ℛ N independent noise for each unit. Motivated by experiments observing phase coding relative to local field potential oscillations (LFPs) [12, 14, 15, 17–19], we used filtered rat CA1 local field potentials [40, 41] as input u ( t ). By using the LFP as input, we make the assumption that the neurons in our model have an overall negligble effect on the LFP, which is valid if they are only a small subset of all contributing neurons, or because the LFP largely reflects input coming an upstream population.…”

“…represent the input weights, and ξ(t) ∈ R N independent noise for each unit. Motivated by experiments observing phase coding relative to local field potential oscillations (LFPs) [12,14,15,[17][18][19], we used filtered rat CA1 local field potentials [40,41] as input u(t). By using the LFP as input, we make the assumption that the neurons in our model have an overall negligble effect on the LFP, which is valid if they are only a small subset of all contributing neurons, or because the LFP largely reflects input coming an upstream population.…”

“…RNNs are universal dynamical systems, in the sense that they can approximate finite time trajectories of any dynamical system [29,30]. We are thus able to use these networks as hypothesis generators -first training them on a cognitive task, and then reverse engineering the resulting networks [17,20,24,26,[31][32][33][34][35][36][37][38].…”

Oscillations in an Artificial Neural Network Convert Competing Inputs into a Temporal Code

“…One mechanism through which the neural representations of multiple objects are organised in time is phase coding, which underlies a modulation of spiking activity by ongoing low-frequency neuronal oscillations [80,93,103] [also see [96,95] for review]. For example, place representations, encoded in spatially distributed firing patterns, have been shown to be ordered along the phase of hippocampal theta (4-8 Hz) oscillations [57,56,61,93,103].…”