Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system

Schmitt, Sebastian; Klähn, Johann; Bellec, Guillaume; Grübl, Andreas; Güttler, Maurice; Hartel, Andreas; Hartmann, Stephan; Husmann, Dan; Husmann, Kai; Jeltsch, Sebastian; Karasenko, Vitali; Kleider, Mitja; Koke, Christoph; Kononov, Alexander; Mauch, Christian; Müller, Eric; Müller, Paul; Partzsch, Johannes; Petrovici, Mihai A.; Schiefer, Stefan; Scholze, Stefan; Thanasoulis, Vasilis; Vogginger, Bernhard; Legenstein, Robert; Maass, Wolfgang; Mayr, Christian; Schüffny, René; Schemmel, Johannes; Meier, K.

doi:10.1109/ijcnn.2017.7966125

Cited by 134 publications

(111 citation statements)

References 3 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…The additional challenge for our neuronal ensemble is to cope with the natural variability of the substrate, caused mainly by fixed-pattern noise, or with other limitations such as a finite weight resolution (4 bits) or spike loss, which can all be substantial [59,60]. It is important to note that the ability to function when embedded in an imperfect substrate with significant deviations from an idealized model represents a necessary prerequisite for viable theories of biological neural function.…”

Section: Implementation On a Neuromorphic Substratementioning

confidence: 99%

“…When endowed with appropriate learning rules, hierarchical spiking networks can be efficiently trained on highdimensional visual data [62,60,47,[63][64][65]. Such hierarchical networks are characterized by the presence of several layers, with connections between consecutive layers, but no lateral connections within the layers themselves.…”

Section: Ensembles Of Hierarchical Ssnsmentioning

confidence: 99%

See 1 more Smart Citation

Stochasticity from function — Why the Bayesian brain may need no noise

et al. 2019

Self Cite

View full text Add to dashboard Cite

An increasing body of evidence suggests that the trial-to-trial variability of spiking activity in the brain is not mere noise, but rather the reflection of a sampling-based encoding scheme for probabilistic computing. Since the precise statistical properties of neural activity are important in this context, many models assume an ad-hoc source of well-behaved, explicit noise, either on the input or on the output side of single neuron dynamics, most often assuming an independent Poisson process in either case. However, these assumptions are somewhat problematic: neighboring neurons tend to share receptive fields, rendering both their input and their output correlated; at the same time, neurons are known to behave largely deterministically, as a function of their membrane potential and conductance. We suggest that spiking neural networks may have no need for noise to perform sampling-based Bayesian inference. We study analytically the effect of auto-and cross-correlations in functional Bayesian spiking networks and demonstrate how their effect translates to synaptic interaction strengths, rendering them controllable through synaptic plasticity. This allows even small ensembles of interconnected deterministic spiking networks to simultaneously and co-dependently shape their output activity through learning, enabling them to perform complex Bayesian computation without any need for noise, which we demonstrate in silico, both in classical simulation and in neuromorphic emulation. These results close a gap between the abstract models and the biology of functionally Bayesian spiking networks, effectively reducing the architectural constraints imposed on physical neural substrates required to perform probabilistic computing, be they biological or artificial. arXiv:1809.08045v3 [q-bio.NC]

show abstract

Section: Implementation On a Neuromorphic Substratementioning

confidence: 99%

Section: Ensembles Of Hierarchical Ssnsmentioning

confidence: 99%

Stochasticity from function — Why the Bayesian brain may need no noise

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Integration (VLSI) ICs and systems that implement hundreds to thousands of spiking neurons and synapses has become an increasing interest of many research groups [1][2][3][4][5]. The electronic models of neurons and synapses are implemented by mimicking biophysically realistic dynamics at different levels of abstraction to meet various hardware implementation constraints.…”

Section: Development Of Brain-inspired Custom Very Large Scalementioning

confidence: 99%

Neuromorphic Architecture for Small-Scale Neocortical Network Emulation

Zhang

Wijekoon

2019

2019 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

The paper presents a neuromorphic platform that can emulate a small-scale cortical network with diverse types of neurons and synapses found in cortical circuits. The platform provides configurable neurons, which behave similarly to the electrophysiological behaviours of different classes of pyramidal and interneurons, and configurable long-and short-term dynamic synapses that can provide inhibition, excitation, weight depressing and facilitating and spike-time dependent plasticity (STDP) dynamics. The prototype of the platform presented in this paper uses a single Cortical Neural Layer (CNL) integrated circuit (IC), which facilitates a network of 120 neurons and 7560 synapses. The number of CNL ICs used in the proposed architecture can be increased to enable larger neural network emulation. The network connectivity is configured using an off-chip Field Programmable Gate Array (FPGA) device. The parameters of the neural elements of the network can be configured using a computer-controlled bias voltages generator. To prove the concept in hardware, Winner-Take-All and Synfire chain networks have been implemented on the platform, and the results are presented.

show abstract

“…By default, synaptic weights on the Spikey chip are only controllable up to 4-bit precision [5]. It is important to note that this does not pose a fundamental problem to networks of this type; the effects of weight discretization can be countered by appropriate in-the-loop training, as discussed in, e.g., [14], [29]. Here, we only take this effect into account as a preparation of the hardware experiments in Sec.…”

Section: Robust Hierarchical Lif Networkmentioning

confidence: 99%

“…It constitutes a perennial challenge for analog neuromorphic system design and operation, and has therefore been often addressed in literature [10]- [13]. A thorough discussion of parameter calibration and in-the-loop training of analog circuits can be found in [14], which represents a complement of the present study. In the present manuscript, we are mainly concerned with the distortions to the network dynamics that are imposed by the physics of the emulation device and that cannot be directly addressed by, e.g., calibration.…”

Section: Introductionmentioning

confidence: 99%

Robustness from structure: Inference with hierarchical spiking networks on analog neuromorphic hardware

Petrovici

Schroeder

Breitwieser

et al. 2017

2017 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Abstract-How spiking networks are able to perform probabilistic inference is an intriguing question, not only for understanding information processing in the brain, but also for transferring these computational principles to neuromorphic silicon circuits. A number of computationally powerful spiking network models have been proposed, but most of them have only been tested, under ideal conditions, in software simulations. Any implementation in an analog, physical system, be it in vivo or in silico, will generally lead to distorted dynamics due to the physical properties of the underlying substrate. In this paper, we discuss several such distortive effects that are difficult or impossible to remove by classical calibration routines or parameter training. We then argue that hierarchical networks of leaky integrate-and-fire neurons can offer the required robustness for physical implementation and demonstrate this with both software simulations and emulation on an accelerated analog neuromorphic device.

show abstract

Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system

Cited by 134 publications

References 3 publications

Stochasticity from function — Why the Bayesian brain may need no noise

Stochasticity from function — Why the Bayesian brain may need no noise

Neuromorphic Architecture for Small-Scale Neocortical Network Emulation

Robustness from structure: Inference with hierarchical spiking networks on analog neuromorphic hardware

Contact Info

Product

Resources

About