Synapse-type-specific competitive Hebbian learning forms functional recurrent networks

Eckmann, Samuel; Gjorgjieva, Julijana

doi:10.1101/2022.03.11.483899

Cited by 4 publications

(15 citation statements)

References 193 publications

(404 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The point at which the feedforward weights converge is fully determined by the covariance matrix of the presynaptic neurons’ activities. Specifically, it has been demonstrated [6, 20, 24] that the fixed points of the weight dynamics are eigenvectors of the modified covariance matrix: where E and I the firing rates of the presynaptic excitatory and inhibitory neurons, respectively, and stands for the average over time, Fig. 1b.…”

Section: Resultsmentioning

confidence: 99%

“…Input selectivity is a universal attribute of brain networks that is maintained across brain hierarchies, including in brain areas that only receive input from highly recurrent networks. Here, we demonstrated that two ubiquitous features of biological networks, namely internal noise and recurrent connectivity between different sub-networks, can impact the statistics of inputs coming from a population in ways that completely prevent known plasticity mechanisms [6, 20, 21, 24] from forming any kind of input selectivity in neurons found in higher areas.…”

Section: Discussionmentioning

confidence: 99%

“…Although we identify different assembly strengths for networks of different sparsity, a general finding is that excitatory assemblies should ubiquitously be stronger than the corresponding inhibitory ones. The formation of E/I assemblies via synaptic plasticity has been extensively studied [20, 24, 25, 28] and it has been demonstrated that a variety of plasticity mechanisms can lead to stable, matching E/I assemblies. Still, the question of different assembly strengths for different connection types has not been fully explored.…”

Section: Discussionmentioning

confidence: 99%

“…Here, we investigate how the development of co-tuned excitation and inhibition via neuronal plasticity is affected by biologically plausible levels of noise and recurrence. We combine excitatory and inhibitory plasticity rules [20, 24, 32, 33] in a spiking network to develop detailed co-tuning of excitatory and inhibitory connectivity. We demonstrate that the ability of these plasticity mechanisms to create co-tuning is significantly reduced in the presence of noise and random recurrent connectivity.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Structural influences on synaptic plasticity: the role of presynaptic connectivity in the emergence of E/I co-tuning

Giannakakis

Vinogradov

Buendía

et al. 2023

Preprint

View full text Add to dashboard Cite

Experimental studies have shown that in cortical neurons, excitatory and inhibitory incoming currents are strongly correlated, which is hypothesized to be essential for efficient computations. Additionally, cortical neurons exhibit strong preference to particular stimuli, which combined with the co-variability of excitatory and inhibitory inputs indicates a detailed co-tuning of the corresponding populations. Such co-tuning is hypothesized to emerge during development in a self-organized manner. Indeed, theoretical studies have demonstrated that a combination of plasticity rules could lead to the emergence of E/I co-tuning in neurons driven by low noise signals from feedforward connections. However, cortical signals are very noisy and originate in highly recurrent networks, which raises a question on the ability of known plasticity mechanisms to self-organize co-tuned connectivity. We demonstrate that high noise levels combined with random recurrence destroy co-tuning. However, we demonstrate that introducing structure in the connectivity patterns of the recurrent E/ I network, the recurrence does not hinder but enhances the formation of the co-tuned selectivity. We employ a combination of analytical methods and simulation-based inference to uncover constraints on the recurrent connectivity that allow E/I co-tuning to emerge. We find that stronger excitatory connectivity within similarly tuned neurons, combined with more homogeneous inhibitory connectivity enhances the ability of plasticity to produce co-tuning in an upstream population. Our results suggest that structured recurrent connectivity controls information propagation and can enhance the ability of synaptic plasticity to learn input-output relationships in higher brain areas.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Structural influences on synaptic plasticity: the role of presynaptic connectivity in the emergence of E/I co-tuning

Giannakakis

Vinogradov

Buendía

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…These properties make SSNs particularly susceptible to dynamical instabilities resulting in run-away excitation, thus rendering their training highly challenging. Indeed, in the few cases in which the training of SSNs was attempted, either noiseless neurons were used [27, 28], or the network was so heavily under-parameterized that it substantially limited its expressivity [29].…”

Section: Introductionmentioning

confidence: 99%

Training stochastic stabilized supralinear networks by dynamics-neutral growth

Soo

Lengyel

2022

Preprint

View full text Add to dashboard Cite

There continues to be a trade-off between the biological realism and performance of neural networks. Contemporary deep learning techniques allow neural networks to be trained to perform challenging computations at (near) human-level, but these networks typically violate key biological constraints. More detailed models of biological neural networks can incorporate many of these constraints but typically suffer from subpar performance and trainability. Here, we narrow this gap by developing an effective method for training a canonical model of cortical neural circuits, the stabilized supralinear network (SSN), that in previous work had to be constructed manually or trained with undue constraints. SSNs are particularly challenging to train for the same reasons that make them biologically realistic: they are characterized by strongly-connected excitatory cells and expansive firing rate non-linearities that together make them prone to dynamical instabilities unless stabilized by appropriately tuned recurrent inhibition. Our method avoids such instabilities by initializing a small network and gradually increasing network size via the dynamics-neutral addition of neurons during training. We first show how SSNs can be trained to perform typical machine learning tasks by training an SSN on MNIST classification. We then demonstrate the effectiveness of our method by training an SSN on the challenging task of performing amortized Markov chain Monte Carlo-based inference under a Gaussian scale mixture generative model of natural image patches with a rich and diverse set of basis functions – something that was not possible with previous methods. These results open the way to training realistic cortical-like neural networks on challenging tasks at scale.

show abstract

Synapse-type-specific competitive Hebbian learning forms functional recurrent networks

Eckmann,

Young,

Gjorgjieva

2024

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Cortical networks exhibit complex stimulus–response patterns that are based on specific recurrent interactions between neurons. For example, the balance between excitatory and inhibitory currents has been identified as a central component of cortical computations. However, it remains unclear how the required synaptic connectivity can emerge in developing circuits where synapses between excitatory and inhibitory neurons are simultaneously plastic. Using theory and modeling, we propose that a wide range of cortical response properties can arise from a single plasticity paradigm that acts simultaneously at all excitatory and inhibitory connections—Hebbian learning that is stabilized by the synapse-type-specific competition for a limited supply of synaptic resources. In plastic recurrent circuits, this competition enables the formation and decorrelation of inhibition-balanced receptive fields. Networks develop an assembly structure with stronger synaptic connections between similarly tuned excitatory and inhibitory neurons and exhibit response normalization and orientation-specific center-surround suppression, reflecting the stimulus statistics during training. These results demonstrate how neurons can self-organize into functional networks and suggest an essential role for synapse-type-specific competitive learning in the development of cortical circuits.

show abstract

Synapse-type-specific competitive Hebbian learning forms functional recurrent networks

Cited by 4 publications

References 193 publications

Structural influences on synaptic plasticity: the role of presynaptic connectivity in the emergence of E/I co-tuning

Structural influences on synaptic plasticity: the role of presynaptic connectivity in the emergence of E/I co-tuning

Training stochastic stabilized supralinear networks by dynamics-neutral growth

Synapse-type-specific competitive Hebbian learning forms functional recurrent networks

Contact Info

Product

Resources

About