Effect of Heterogeneity on Decorrelation Mechanisms in Spiking Neural Networks: A Neuromorphic-Hardware Study

Abstract

High-level brain function such as memory, classification or reasoning can be realized by means of recurrent networks of simplified model neurons. Analog neuromorphic hardware constitutes a fast and energy efficient substrate for the implementation of such neural computing architectures in technical applications and neuroscientific research. The functional performance of neural networks is often critically dependent on the level of correlations in the neural activity. In finite networks, correlations are typically inevitable due to shared presynaptic input. Recent theoretical studies have shown that inhibitory feedback, abundant in biological neural networks, can actively suppress these shared-input correlations and thereby enable neurons to fire nearly independently. For networks of spiking neurons, the decorrelating effect of inhibitory feedback has so far been explicitly demonstrated only for homogeneous networks of neurons with linear sub-threshold dynamics. Theory, however, suggests that the effect is a general phenomenon, present in any system with sufficient inhibitory feedback, irrespective of the details of the network structure or the neuronal and synaptic properties. Here, we investigate the effect of network heterogeneity on correlations in sparse, random networks of inhibitory neurons with non-linear, conductance-based synapses. Emulations of these networks on the analog neuromorphic hardware system Spikey allow us to test the efficiency of decorrelation by inhibitory feedback in the presence of hardware-specific heterogeneities. The configurability of the hardware substrate enables us to modulate the extent of heterogeneity in a systematic manner. We selectively study the effects of shared input and recurrent connections on correlations in membrane potentials and spike trains. Our results confirm that shared-input correlations are actively suppressed by inhibitory feedback also in highly heterogeneous networks exhibiting broad, heavy-tailed firing-rate distributions. In line with former studies, cell heterogeneities reduce shared-input correlations. Overall, however, correlations in the recurrent system can increase with the level of heterogeneity as a consequence of diminished effective negative feedback.

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

“…An extension to networks of leaky integrate-and-fire neurons and a theoretical framework for their dynamics and statistics followed in Petrovici et al (2013) and Petrovici et al (2016). The compensation of shared-input correlations through inhibitory feedback and learning was discussed in Bytschok et al (2017), Jordan et al (2017), and Dold et al (2019), inspired by the early study of asynchronous irregular firing in Brunel (2000) and by preceding correlation studies in theoretical (Tetzlaff et al, 2012) and experimental (Pfeil et al, 2016) work.…”

Deterministic networks for probabilistic computing

“…It solely relies on the capability of emulating recurrent neural networks, the functionality most neuromorphic-hardware systems are designed for. On the analog neuromorphic system Spikey 67 , for example, it has already been demonstrated that decorrelation by inhibitory feedback is effective and robust, despite large heterogeneity in neuron and synapse parameters and without the need for time-consuming calibrations 68 . While a full neuromorphic-hardware implementation of the framework proposed here is still pending, the demonstration on Spikey shows that our solution is immediately implementable and feasible.…”

Quantum computing with classical bits

“…For example, performing first the transformation U X yields ρ 2 = −ρ 2 , ρ 3 = −ρ 3 . Applying subsequently the transformation U 31 , ρ 1 = −ρ 3 = ρ 3 , ρ 3 = ρ 1 = ρ 1 , ρ 2 = ρ 2 = −ρ 2 , results in the transformation (12), as reflected by the corresponding matrix multiplication yielding eq. (11),…”

“…It is straightforward to show that the CNOT-gate cannot be realized on the operator level. A realization on the operator level would require relations of the type (39) on the level of the classical operators B (µν) and A (µν) , which amounts to [S, A (30) ] = [S, A (01) ] = [S, A (31) ] = 0 , SA (11) = A (10) S , SA (10) = A (11) S , SA (21) = A (20) S , SA (20) = A (21) S , SA (33) = A (03) S , SA (03) = A (33) S , SA (32) = A (02) S , SA (02) = A (32) S , SA (22) = −A (13) S , SA (13) = −A (22) S , SA (12) = A (23) S , SA (23) = A (12) S .…”

“…Since neural networks operate classically, this amounts to realizing quantum statistics by classical statistical systems. Neuromorphic computing [11][12][13][14][15][16][17], or computational steps in our brain, may also realize quantum operations based on classical statistics. Learning by evolution, living systems may be able to perform quantum computational steps without realizing conditions that are often assumed to be necessary for quantum mechanics, as the presence of small and well isolated systems or low temperature.…”