Efficient Spiking Neural Networks With Logarithmic Temporal Coding

Zhang, Ming; Gu, Zonghua; Zheng, Nenggan; De, Ma; Pan, Gang

doi:10.1109/access.2020.2994360

Cited by 9 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another direction is to use temporal instead of rate-based spike coding to represent ANN activities. Zhang et al [128] have shown that temporal coding using a logarithmic time scale is the most efficient, second only to time-to-first-spike (TTFS) [129] coding. However, compared to TTFS, the logarithmic time scale approach is more compatible with popular deep learning techniques.…”

Section: O U T L O O K a Deep Networkmentioning

confidence: 99%

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

et al. 2021

View full text Add to dashboard Cite

Deep artificial neural networks apply principles of the brain's information processing that led to breakthroughs in machine learning spanning many problem domains. Neuromorphic computing aims to take this a step further to chips more directly inspired by the form and function of biological neural circuits, so they can process new knowledge, adapt, behave, and learn in real time at low power levels. Despite several decades of research, until recently, very few published results have shown that today's neuromorphic chips can demonstrate quantitative computational value. This is now changing with the advent of Intel's Loihi, a neuromorphic research processor designed to support a broad range of spiking neural networks with sufficient scale, performance, and features to deliver competitive results compared to state-of-the-art contemporary computing architectures. This survey reviews results that are obtained to date with Loihi across the major algorithmic domains under study, including deep learning approaches and novel approaches that aim to more directly harness the key features of spike-based neuromorphic hardware. While conventional feedforward deep neural networks show modest if any benefit on Loihi, more brain-inspired networks using recurrence, precise spike-timing relationships, synaptic plasticity, stochasticity, and sparsity perform certain computation with orders of magnitude lower latency and energy compared to state-of-the-art conventional approaches. These compelling neuromorphic networks solve a diverse range of problems representative of brain-like computation, such as event-based

show abstract

Section: O U T L O O K a Deep Networkmentioning

confidence: 99%

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A different approach uses a global referenced binary coding to reduce the number of spikes. Together with neuron models with exponential input characteristics, the same activation of the neuron can be reached as with a count rate code but with far less spikes [110].…”

Section: Rate Codingmentioning

confidence: 95%

“…Here, each spike corresponds to a "1" or "0" in a bit stream. In relation to a fixed reference clock, two schemes to encode the bits are possible: the presence or absence of a spike within a given interval [110], or the timing of the spike within the interval [33]. In the former case, a logical "1" corresponds to a spike being present during one clock cycle.…”

Section: Global Referencedmentioning

confidence: 99%

A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks

Auge

Hille

Mueller

et al. 2021

Neural Process Lett

109

View full text Add to dashboard Cite

Biologically inspired spiking neural networks are increasingly popular in the field of artificial intelligence due to their ability to solve complex problems while being power efficient. They do so by leveraging the timing of discrete spikes as main information carrier. Though, industrial applications are still lacking, partially because the question of how to encode incoming data into discrete spike events cannot be uniformly answered. In this paper, we summarise the signal encoding schemes presented in the literature and propose a uniform nomenclature to prevent the vague usage of ambiguous definitions. Therefore we survey both, the theoretical foundations as well as applications of the encoding schemes. This work provides a foundation in spiking signal encoding and gives an overview over different application-oriented implementations which utilise the schemes.

show abstract

“…The other coding scheme is phase (or binary) coding that combines the advantages of both rate and temporal coding [20], [21], [29]. The phase coding assigns different weights w t to each time segment t within T so that activation values are encoded by both the number of spikes and the spike pattern.…”

Section: A Activation Encoding In Ann-to-snn Conversionmentioning

confidence: 99%

“…Each spike at t has a phase Q −t where Q = 2 for binary coding. [29]. The latter is used to realize the phase coding in this paper.…”

Section: B Neuron Modelmentioning

confidence: 99%

Comparison and Selection of Spike Encoding Algorithms for SNN on FPGA

Wang

Hao

Wang

et al. 2023

IEEE Trans. Biomed. Circuits Syst.

View full text Add to dashboard Cite

As spiking neural networks (SNNs) are event-driven, energy efficiency is higher than conventional artificial neural networks (ANNs). Since SNN delivers data through discrete spikes, it is difficult to use gradient methods for training, limiting its accuracy. To keep the accuracy of SNNs similar to ANN counterparts, pre-trained ANNs are converted to SNNs (ANNto-SNN conversion). During the conversion, encoding activations of ANNs to a set of spikes in SNNs is crucial for minimizing the conversion loss. In this work, we propose a single-spike phase coding as an encoding scheme that minimizes the number of spikes to transfer data between SNN layers. To minimize the encoding error due to single-spike approximation in phase coding, threshold shift and base manipulation are proposed. Without any additional retraining or architectural constraints on ANNs, the proposed conversion method does not lose inference accuracy (0.58% on average) verified on three convolutional neural networks (CNNs) with CIFAR and ImageNet datasets. In addition, graph convolutional networks (GCNs) are converted to SNNs successfully with an average accuracy loss of 0.90%. Most importantly, the energy efficiency of our SNN improves by 4.6∼17.3× compared to the ANN baseline.

show abstract

Efficient Spiking Neural Networks With Logarithmic Temporal Coding

Cited by 9 publications

References 20 publications

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks

Comparison and Selection of Spike Encoding Algorithms for SNN on FPGA

Contact Info

Product

Resources

About