Stochastic Computing Can Improve Upon Digital Spiking Neural Networks

Smithson, Sean C.; Boga, Kaushik; Ardakani, Arash; Meyer, Brett H.; Gross, Warren J.

doi:10.1109/sips.2016.61

Cited by 23 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed algorithms can be extended to other spiking and non-spiking hardware substrates that have the ability to perform stochastic computing and provides the capability to have recurrent neural network connections. Prior work such as Smithson et al ( 2016 ), Thakur et al ( 2016 ), and Cassidy et al ( 2013 ) show that digital spiking neural substrates can perform stochastic computing. We can also perform stochastic computing on non-spiking hardware substrates such as FPGAs (Li et al, 2016 ), FinFETs (Zhang et al, 2017 ), and magnetic-tunnel-junction (Lv and Wang, 2017 ).…”

Section: Discussionmentioning

confidence: 99%

Computing Generalized Matrix Inverse on Spiking Neural Substrate

et al. 2018

View full text Add to dashboard Cite

Emerging neural hardware substrates, such as IBM's TrueNorth Neurosynaptic System, can provide an appealing platform for deploying numerical algorithms. For example, a recurrent Hopfield neural network can be used to find the Moore-Penrose generalized inverse of a matrix, thus enabling a broad class of linear optimizations to be solved efficiently, at low energy cost. However, deploying numerical algorithms on hardware platforms that severely limit the range and precision of representation for numeric quantities can be quite challenging. This paper discusses these challenges and proposes a rigorous mathematical framework for reasoning about range and precision on such substrates. The paper derives techniques for normalizing inputs and properly quantizing synaptic weights originating from arbitrary systems of linear equations, so that solvers for those systems can be implemented in a provably correct manner on hardware-constrained neural substrates. The analytical model is empirically validated on the IBM TrueNorth platform, and results show that the guarantees provided by the framework for range and precision hold under experimental conditions. Experiments with optical flow demonstrate the energy benefits of deploying a reduced-precision and energy-efficient generalized matrix inverse engine on the IBM TrueNorth platform, reflecting 10× to 100× improvement over FPGA and ARM core baselines.

show abstract

Section: Discussionmentioning

confidence: 99%

Computing Generalized Matrix Inverse on Spiking Neural Substrate

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Stochastic computation techniques apply randomness to the process of computation itself (Shanbhag et al, 2010). Variants of this approach have been applied to spiking neural networks (Rosselló et al, 2012;Ahmed et al, 2016;Smithson et al, 2016). These are mostly orthogonal to the ideas we discuss, since a different (stochastic) hardware architecture for elementary compute units can also be incorporated into our approach which introduces randomness in the process of spike propagation.…”

Section: Stochastic Techniquesmentioning

confidence: 99%

Probabilistic Spike Propagation for Efficient Hardware Implementation of Spiking Neural Networks

et al. 2021

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have gained considerable attention in recent years due to their ability to model temporal event streams, be trained using unsupervised learning rules, and be realized on low-power event-driven hardware. Notwithstanding the intrinsic desirable attributes of SNNs, there is a need to further optimize their computational efficiency to enable their deployment in highly resource-constrained systems. The complexity of evaluating an SNN is strongly correlated to the spiking activity in the network, and can be measured in terms of a fundamental unit of computation, viz. spike propagation along a synapse from a single source neuron to a single target neuron. We propose probabilistic spike propagation, an approach to optimize rate-coded SNNs by interpreting synaptic weights as probabilities, and utilizing these probabilities to regulate spike propagation. The approach results in 2.4–3.69× reduction in spikes propagated, leading to reduced time and energy consumption. We propose Probabilistic Spiking Neural Network Application Processor (P-SNNAP), a specialized SNN accelerator with support for probabilistic spike propagation. Our evaluations across a suite of benchmark SNNs demonstrate that probabilistic spike propagation results in 1.39–2× energy reduction with simultaneous speedups of 1.16–1.62× compared to the traditional model of SNN evaluation.

show abstract

“…Despite the great similarities between SNNs and SC-based neural networks in inference computations [99], the training procedure of SNNs is significantly different from that of SCbased neural networks. SNNs are trained using an unsupervised biological learning method called spike-timing-dependent plasticity (STDP), where weights are updated depending on the relative timing of incoming and outgoing spikes of neurons [100].…”

Section: A Neuromorphic Computingmentioning

confidence: 99%