Nonlinear transient amplification in recurrent neural networks with short-term plasticity

Abstract

To rapidly process information, neural circuits have to amplify specific activity patterns transiently. How the brain performs this nonlinear operation remains elusive. Hebbian assemblies are one possibility whereby strong recurrent excitatory connections boost neuronal activity. However, such Hebbian amplification is often associated with dynamical slowing of network dynamics, non-transient attractor states, and pathological run-away activity. Feedback inhibition can alleviate these effects but typically linearizes responses and reduces amplification gain. Here we study nonlinear transient amplification (NTA), a plausible alternative mechanism that reconciles strong recurrent excitation with rapid amplification while avoiding the above issues. NTA has two distinct temporal phases. Initially, positive feedback excitation selectively amplifies inputs that exceed a critical threshold. Subsequently, short-term plasticity quenches the run-away dynamics into an inhibition-stabilized network state. By characterizing NTA in supralinear network models, we establish that the resulting onset transients are stimulus selective and well-suited for speedy information processing. Further, we find that excitatory-inhibitory co-tuning widens the parameter regime in which NTA is possible in the absence of persistent activity. In summary, NTA provides a parsimonious explanation for how excitatory-inhibitory co-tuning and short-term plasticity collaborate in recurrent networks to achieve transient amplification.

Reduced variability of bursting activity during working memory

“…Further, recent evidence has shown that the short-term synaptic plasticity endows additional benefits beyond memory maintenance. For instance, it helps make recurrent networks more stable and more robust to perturbations and synaptic loss (Kozachkov et al, 2022; Wu and Zenke, 2021).…”

Targeting diverse operational regimes in recurrent spiking networks

“…Previous work identified the constraints on connectivity configurations in the SSN model that underlie such nonlinear activity responses as supersaturation [15], the paradoxical effect [47, 48], bistability, and persistent activity [28]. We show that the parameters of LIF spiking networks can be mapped to the SSN such that the same activity types emerge in the spiking network, according to the observations made with the SSN.…”

“…A recent review presented further experimental evidence and techniques used to study the inhibition-stabilized dynamics and discussed the ISN consequences for cortical computation [29]. In the SSN model [47, 48], a network is inhibition-stabilized if it fulfills the condition We note that in networks with a threshold linear transfer function, the analogous ISN condition only requires a strong recurrent coupling J EE > 1 and does not impose any constraints on the E firing rate level or the transfer function parameters [25, 29]. However, large enough J EE does not always guarantee that a recurrent neural network with a nonlinear transfer function is in the ISN regime.…”

“…However, other features, such as strong feedforward inputs, have not been experimentally reported, and some experiments argue against it [22, 64–67]. Unlike the balanced state framework, which assumes a tight E/I input balance, alternative models such as the SSN only assume loose E/I balance to achieve inhibitory stabilization [20, 29, 48, 57, 58]. Yet, clarifying under which conditions the predictions of these two models are conflicting or aligning and how they compare to spiking network simulations has so far remained an open question.…”

“…The paradoxical effect [25, 47, 48] is characterized by a decrease of the I firing rate, as the external input to the I population is increased: . Here again, the effect of a change in external input is given by Eq.…”