Stable Control of Firing Rate Mean and Variance by Dual Homeostatic Mechanisms

Cannon, Jonathan; Miller, Paul

doi:10.1186/s13408-017-0043-7

Cited by 29 publications

(48 citation statements)

References 14 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For intrinsic learning it has often been assumed that it may implement purely homeostatic adaptation [26,27,28,29,30], but see also [31]. Experimental results are often inconsistent [32,33,34,4,35,36,37,38].…”

Section: Introductionmentioning

confidence: 99%

Logarithmic distributions prove that intrinsic learning is Hebbian

Scheler¹

2017

F1000Res

View full text Add to dashboard Cite

In this paper, we document lognormal distributions for spike rates, synaptic weights and intrinsic excitability (gain) for neurons in various brain areas, such as auditory or visual cortex, hippocampus, cerebellum, striatum, midbrain nuclei. We find a remarkable consistency of heavy-tailed, specifically lognormal, distributions for rates, weights and gains in all brain areas. The difference between strongly recurrent and feed-forward connectivity (cortex vs. striatum and cerebellum), neurotransmitter (GABA (striatum) or glutamate (cortex)) or the level of activation (low in cortex, high in Purkinje cells and midbrain nuclei) turns out to be irrelevant for this feature. Logarithmic scale distribution of weights and gains appears as a functional property that is present everywhere. Secondly, we created a generic neural model to show that Hebbian learning will create and maintain lognormal distributions. We could prove with the model that not only weights, but also intrinsic gains, need to have strong Hebbian learning in order to produce and maintain the experimentally attested distributions. This settles a long-standing question about the type of plasticity exhibited by intrinsic excitability.

show abstract

Section: Introductionmentioning

confidence: 99%

Logarithmic distributions prove that intrinsic learning is Hebbian

Scheler¹

2017

F1000Res

View full text Add to dashboard Cite

show abstract

“…The mechanisms act on two sets of node-specific parameters, the biases b i and the neural gain factors a i . This approach can be considered a realization of dual homeostasis, which has been investigated previously with respect to a stable control of the mean and the variance of neural activity [26]. In this framework, the adaptation of the bias acts an intrinsic plasticity for the control of the internal excitability of a neuron [27][28][29], while the gain factors functionally correspond to a synaptic scaling of the recurrent weights [30][31][32].…”

Section: An Extension To Layered Esn Architectures Was Presented By Gmentioning

confidence: 99%

“…is obtained when using R t = 1 in (16), and σ w = 1 for the normalized variance of the synaptic weights, as defined by (26). We used the mean-field approximation for neural variances that is derived in in S2 Appendix.…”

Section: Spectral Radius Adaption Dynamicsmentioning

confidence: 99%

Local homeostatic regulation of the spectral radius of echo-state networks

Schubert

Gros

2020

Preprint

View full text Add to dashboard Cite

Recurrent cortical network dynamics plays a crucial role for sequential information processing in the brain. While the theoretical framework of reservoir computing provides a conceptual basis for the understanding of recurrent neural computation, it often requires manual adjustments of global network parameters, in particular of the spectral radius of the recurrent synaptic weight matrix. Being a mathematical and relatively complex quantity, the spectral radius is not readily accessible to biological neural networks, which generally adhere to the principle that information about the network state should either be encoded in local intrinsic dynamical quantities (e.g. membrane potentials), or transmitted via synaptic connectivity. We present two synaptic scaling rules for echo state networks that solely rely on locally accessible variables. Both rules work online, in the presence of a continuous stream of input signals. The first rule, termed flow control, is based on a local comparison between the mean squared recurrent membrane potential and the mean squared activity of the neuron itself. It is derived from a global scaling condition on the dynamic flow of neural activities and requires the separability of external and recurrent input currents. We gained further insight into the adaptation dynamics of flow control by using a mean field approximation on the variances of neural activities that allowed us to describe the interplay between network activity and adaptation as a two-dimensional dynamical system. The second rule that we considered, variance control, directly regulates the variance of neural activities by locally scaling the recurrent synaptic weights. The target set point of this homeostatic mechanism is dynamically determined as a function of the variance of the locally measured external input. This functional relation was derived from the same mean-field approach that was used to describe the approximate dynamics of flow control.The effectiveness of the presented mechanisms was tested numerically using different external input protocols. The network performance after adaptation was evaluated by training the network to perform a time delayed XOR operation on binary sequences. As our main result, we found that flow control can reliably regulate the spectral radius under different input statistics, but precise tuning is negatively affected by interneural correlations. Furthermore, flow control showed a consistent task performance over a wide range of input strengths/variances. Variance control, on the other side, did not yield the desired spectral radii with the same precision. Moreover, task performance was less consistent across different input strengths.Given the better performance and simpler mathematical form of flow control, we concluded that a local control of the spectral radius via an implicit adaptation scheme is a realistic alternative to approaches using classical “set point” homeostatic feedback controls of neural firing.Author summaryHow can a neural network control its recurrent synaptic strengths such that network dynamics are optimal for sequential information processing? An important quantity in this respect, the spectral radius of the recurrent synaptic weight matrix, is a non-local quantity. Therefore, a direct calculation of the spectral radius is not feasible for biological networks. However, we show that there exist a local and biologically plausible adaptation mechanism, flow control, which allows to control the recurrent weight spectral radius while the network is operating under the influence of external inputs. Flow control is based on a theorem of random matrix theory, which is applicable if inter-synaptic correlations are weak. We apply the new adaption rule to echo-state networks having the task to perform a time-delayed XOR operation on random binary input sequences. We find that flow-controlled networks can adapt to a wide range of input strengths while retaining essentially constant task performance.

show abstract

“…The interaction between homeostasis and oscillations has previously been considered when the oscillatory input is treated as a signal, not a modulator. Cannon and Miller [7] explored how synaptic homeostasis can effectively minimize the effect of modulatory perturbations, thus maximizing mutual information between an incoming oscillatory signal and a single cell's firing pattern.…”

Section: Previous Workmentioning

confidence: 99%

“…Our analysis could be considered an inverse complement to [7]-we study how to minimize the perturbation caused by a modulatory oscillator, rather than how to maximize the transmission of an oscillator.…”

Section: Previous Workmentioning

confidence: 99%

Homeostatic mechanisms may shape the type and duration of oscillatory modulation

Peterson

Voytek

2019

Preprint

View full text Add to dashboard Cite

Neural oscillations are observed ubiquitously in the mammalian brain. However the stability of oscillations is highly variable. Some oscillations are tonic, lasting for seconds or even minutes; others are unstable, appearing only as a single-cycle burst. In a model of hippocampal neurons, we use numerical simulations to show how these different forms of rhythm stability can interact with activity-dependent homeostasis to profoundly alter the modulatory effect of neural oscillations. Under homeostasis, tonic oscillations that are synaptically excitatory have a paradoxical effect; they decrease excitability and desynchronizing firing. Tonic oscillations that are synaptically inhibitory-like those in a real hippocampus-fail to generate new action potentials and so provoke no homeostatic response. This may explain why the theta rhythm in hippocampus is synaptically inhibitory: inhibitory oscillations don't raise the firing threshold, as excitatory oscillations do, and so can preserve each cell's dynamic range. Based on these simulations, we also speculate that homeostasis may explain why excitatory intra-cortical and intra-layer oscillations often appear as bursts. In our model bursts minimally interact with the slow homeostasis time constant and so retain typical excitatory effects.

show abstract

Stable Control of Firing Rate Mean and Variance by Dual Homeostatic Mechanisms

Cited by 29 publications

References 14 publications

Logarithmic distributions prove that intrinsic learning is Hebbian

Logarithmic distributions prove that intrinsic learning is Hebbian

Local homeostatic regulation of the spectral radius of echo-state networks

Homeostatic mechanisms may shape the type and duration of oscillatory modulation

Contact Info

Product

Resources

About