Input correlations impede suppression of chaos and learning in balanced firing-rate networks

Engelken, Rainer; Ingrosso, Alessandro; Khajeh, Ramin; Goedeke, Sven; Abbott, L. F.

doi:10.1371/journal.pcbi.1010590

Cited by 6 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neuronal network architectures similar to the OSP have been widely used for different purposes in the context of machine learning [8] and computational neuroscience [15,59,62], with nonlinear neurons governed by bounded sigmoid-like profiles like in the Wilson-Cowanmodel [69,70], or rectified linear units (ReLU) [45]. [55] give a thorough review of different gain functions and their influence on computational performance.…”

Section: Discussionmentioning

confidence: 99%

Neuronal architecture extracts statistical temporal patterns

Nestler¹,

Helias²,

Gilson³

2023

Preprint

View full text Add to dashboard Cite

Neuronal systems need to process temporal signals. We here show how higher-order temporal (co-)fluctuations can be employed to represent and process information. Concretely, we demonstrate that a simple biologically inspired feedforward neuronal model is able to extract information from up to the third order cumulant to perform time series classification. This model relies on a weighted linear summation of synaptic inputs followed by a nonlinear gain function. Training both -the synaptic weights and the nonlinear gain function -exposes how the non-linearity allows for the transfer of higher order correlations to the mean, which in turn enables the synergistic use of information encoded in multiple cumulants to maximize the classification accuracy. The approach is demonstrated both on a synthetic and on real world datasets of multivariate time series. Moreover, we show that the biologically inspired architecture makes better use of the number of trainable parameters as compared to a classical machine-learning scheme. Our findings emphasize the benefit of biological neuronal architectures, paired with dedicated learning algorithms, for the processing of information embedded in higher-order statistical cumulants of temporal (co-)fluctuations.

show abstract

Section: Discussionmentioning

confidence: 99%

Neuronal architecture extracts statistical temporal patterns

Nestler¹,

Helias²,

Gilson³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…They have developed a method called FORCE that allows the reproduction of complex output patterns, including human motion-captured data (Sussillo and Abbott ( 2009 )). Modifications to the algorithm have also been applied successfully in various applications (DePasquale et al, 2018 ; Ingrosso and Abbott, 2019 ; Engelken et al, 2022 ).…”

Section: Training Protocol and Parameter Selectionmentioning

confidence: 99%

Exploring Flip Flop memories and beyond: training Recurrent Neural Networks with key insights

Jarne

2024

Front. Syst. Neurosci.

View full text Add to dashboard Cite

Training neural networks to perform different tasks is relevant across various disciplines. In particular, Recurrent Neural Networks (RNNs) are of great interest in Computational Neuroscience. Open-source frameworks dedicated to Machine Learning, such as Tensorflow and Keras have produced significant changes in the development of technologies that we currently use. This work contributes by comprehensively investigating and describing the application of RNNs for temporal processing through a study of a 3-bit Flip Flop memory implementation. We delve into the entire modeling process, encompassing equations, task parametrization, and software development. The obtained networks are meticulously analyzed to elucidate dynamics, aided by an array of visualization and analysis tools. Moreover, the provided code is versatile enough to facilitate the modeling of diverse tasks and systems. Furthermore, we present how memory states can be efficiently stored in the vertices of a cube in the dimensionally reduced space, supplementing previous results with a distinct approach.

show abstract

Boosting of neural circuit chaos at the onset of collective oscillations

Palmigiano¹,

Engelken²,

Wolf

2022

Preprint

View full text Add to dashboard Cite

Cortical networks exhibit highly irregular spiking activity to repeated stimuli. One source of such variability is network chaos, ever-present in spiking network models. In order to reliably and robustly represent and transmit information, cortical networks must implement active mechanisms to quench undesired changes in spiking patterns. Neuronal oscillations have been proposed as a mechanism to flexibly gate information flow across the cortex, but whether these collective rhythms would actually contribute to quenching spiking unreliability and tame network chaos by synchronizing activity, or on the contrary, intensify chaos by acting as a common drive to the network, is not easily predictable. Here we investigate the dynamical properties of network models with respect to two known control parameters regulating collective oscillatory activity: delayed recurrent inhibition and an external periodic drive. To do so, we advanced the tractability of large spiking networks of exactly solvable neuronal models by developing a strategy that allows for exact characterization of the dynamics on the attractor of effectively delayed network models in a system with fixed and finite degrees of freedom. We find that, below the transition to collective oscillations, neuronal networks have a stereotypical dependence on the delay so far only described for scalar systems and low-dimensional maps. We demonstrate that the emergence of internally generated oscillations induces a complete dynamical reconfiguration, by increasing the dimensionality of the chaotic attractor, the speed at which nearby trajectories separate from one another, and the rate at which the network produces entropy. Our results suggest that simple temporal dynamics of the mean activity can have a profound effect on the structure of the spiking patterns and therefore on the information processing capability of neuronal networks.

show abstract

Input correlations impede suppression of chaos and learning in balanced firing-rate networks

Cited by 6 publications

References 36 publications

Neuronal architecture extracts statistical temporal patterns

Neuronal architecture extracts statistical temporal patterns

Exploring Flip Flop memories and beyond: training Recurrent Neural Networks with key insights

Boosting of neural circuit chaos at the onset of collective oscillations

Contact Info

Product

Resources

About