Oscillations in an Artificial Neural Network Convert Competing Inputs into a Temporal Code

Duecker, Katharina; Idiart, Marco; van Gerven, Marcel AJ; Jensen, Ole

doi:10.1101/2023.11.27.568876

Cited by 3 publications

(3 citation statements)

References 122 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to study the dynamics of phase coding during working memory, we defined a task in which an RNN receives transient stimuli, and has to encode their identity using the relative phase of oscillations (Fig 1A). The network consists of N units, with activation xðtÞ 2 R N , recurrently connected via a connectivity matrix J 2 R N�N , and receiving external oscillatory input uðtÞ 2 R [7,18,39], as well as stimuli sðtÞ 2 R 2 ,…”

Section: Tractable Oscillating Recurrent Neural Network Perform a Wor...mentioning

confidence: 99%

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals,

Macke,

Barak

2024

PLoS Comput Biol

View full text Add to dashboard Cite

Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

show abstract

Section: Tractable Oscillating Recurrent Neural Network Perform a Wor...mentioning

confidence: 99%

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals,

Macke,

Barak

2024

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…In order to study the dynamics of phase coding during working memory, we defined a task in which an RNN receives transient stimuli, and has to encode their identity using the relative phase of oscillations (Fig 1A). The network consists of N units, with activation x(t) ∈ R N , recurrently connected via a connectivity matrix J ∈ R N ×N , and receiving external oscillatory input u(t) ∈ R [7,18,39], as well as stimuli s(t) ∈ R 2 ,…”

Section: Tractable Oscillating Recurrent Neural Network Perform a Wor...mentioning

confidence: 99%

“…1A). The network consists of N units, with activation x(t) ∈ R N , recurrently connected via a connectivity matrix J ∈ R N ×N , and receiving external oscillatory input u(t) ∈ R [17,37,38], as well as stimuli s(t) ∈ R 2 ,…”

Section: Tractable Oscillating Recurrent Neural Network Perform a Wor...mentioning

confidence: 99%

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals

Macke

Barak

2023

Preprint

View full text Add to dashboard Cite

Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or 'frame of reference'. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

show abstract

Theta oscillations optimize a speed-precision trade-off in phase coding neurons

Amil

Albesa-González

Verschure

2022

Preprint

View full text Add to dashboard Cite

Low-frequency oscillations shape how neurons sample their synaptic inputs, hence regulating information exchange among neural circuits. In the hippocampus, theta oscillations (4-8 Hz) enable the temporal organization of cortical inputs, resulting in a phase code. However, the advantages of the specific theta band over other frequency ranges remain unclear. It is possible that this specific frequency range is optimizing a trade-off between information throughput and biophysical constraints of the neuronal substrate. To test this hypothesis, we analyze a physiologically constrained model of the rodent hippocampus comprising stochastic leaky integrate-and-fire neurons driven by oscillations. By estimating the information rate of the phase code for a range of noise and frequency levels, we identify a trade-off between sampling frequency and coding precision. We observe that, under physiological noise levels, theta-band oscillations optimize the speed-precision trade-off, maximizing information rate. Further, we demonstrate that theta optimizes this trade-off throughout the entire dorsoventral axis of the hippocampus with its pronounced gradients of several physiological properties. In addition, we find that maintaining an optimal information rate relies on the concurrent modulation of both frequency and amplitude, hence explaining the locomotion speed-dependent modulations of the hippocampal theta oscillation that are observed in rodents and humans. Overall, our results suggest that low-frequency oscillations are adapted to maximize the information rate of neural sampling given the temporal and biophysical constraints under which neural circuits operate.

show abstract

Oscillations in an Artificial Neural Network Convert Competing Inputs into a Temporal Code

Cited by 3 publications

References 122 publications

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Theta oscillations optimize a speed-precision trade-off in phase coding neurons

Contact Info

Product

Resources

About