A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design

Sakemi, Yusuke; Morino, Kazuhide; Matsukawa, Takashi; Aihara, Kazuyuki

doi:10.48550/arxiv.2001.05348

Cited by 3 publications

(8 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Oh et al designed analog SNN circuits with analog memory based on TTFS coding [17]. Although Kheradpishesh et al [9] and Sakemi et al [19] investigated the effects of the timing jitter of spikes on the performance, the effects of the time discretization of the spikes have not been discussed. Sakemi et al [19] and Oh et al [17] investigated the effects of fixed scattering of the firing thresholds owing to device mismatches.…”

Section: Related Workmentioning

confidence: 99%

“…Although Kheradpishesh et al [9] and Sakemi et al [19] investigated the effects of the timing jitter of spikes on the performance, the effects of the time discretization of the spikes have not been discussed. Sakemi et al [19] and Oh et al [17] investigated the effects of fixed scattering of the firing thresholds owing to device mismatches. However, they did not discuss the effects of the temporal fluctuation of the firing thresholds.…”

Section: Related Workmentioning

confidence: 99%

“…To improve the generalization ability, Gaussian noise was introduced into the timing of the input spikes in the training phase. The SNNs were trained with the backpropagation algorithm described in [19]. We investigated the following effects in the VLSI implementation of SNNs: (i) the discretization of the spike timing of the neurons, (ii) the quantization of the weights, (iii) the lower bound of the membrane potentials, and (iv) the temporal fluctuation of the firing threshold.…”

Section: Modelsmentioning

confidence: 99%

“…We consider an integrate-and-fire model as illustrated in Fig. 6, which is commonly employed in matrix-vector computation [23][22][1] [25] and SNNs [19,17]. The time evolution of the membrane potential represented by the voltage at a capacitor is obtained by Kirchhoff's current law, as follows: where I (min) represents the controllable minimum current, I (level) ij is an integer, and C is the capacitance.…”

Section: Circuit Modelmentioning

confidence: 99%

“…Although rate coding is closely linked to ANNs [5] [11] in which the neuron states are represented by analog values, temporal coding can efficiently exploit the information of the temporal dynamics of the membrane potentials and synaptic currents, which are not considered in ANNs [7]. In particular, time-to-first-spike (TTFS) coding, which is a type of temporal coding, is expected to be the most efficient because the information of a neuron output is represented as the timing of a single spike [2] [16][3] [19].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks

Sakemi¹,

Matsukawa²,

Hosomi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The spiking neural network (SNN) has been attracting considerable attention not only as a mathematical model for the brain, but also as an energy-efficient information processing model for real-world applications. In particular, SNNs based on temporal coding are expected to be much more efficient than those based on rate coding, because the former requires substantially fewer spikes to carry out tasks. As SNNs are continuous-state and continuous-time models, it is favorable to implement them with analog VLSI circuits. However, the construction of the entire system with continuous-time analog circuits would be infeasible when the system size is very large. Therefore, mixed-signal circuits must be employed, and the time discretization and quantization of the synaptic weights are necessary. Moreover, the analog VLSI implementation of SNNs exhibits non-idealities, such as the effects of noise and device mismatches, as well as other constraints arising from the analog circuit operation. In this study, we investigated the effects of the time discretization and/or weight quantization on the performance of SNNs. Furthermore, we elucidated the effects the lower bound of the membrane potentials and the temporal fluctuation of the firing threshold. Finally, we propose an optimal approach for the mapping of mathematical SNN models to analog circuits with discretized time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Modelsmentioning

confidence: 99%

Section: Circuit Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks

Sakemi¹,

Matsukawa²,

Hosomi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Monitoring time domain characteristics of Parkinson’s disease using 3D memristive neuromorphic system

Siddique,

Zhang,

2023

Front. Comput. Neurosci.

View full text Add to dashboard Cite

IntroductionParkinson’s disease (PD) is a neurodegenerative disorder affecting millions of patients. Closed-Loop Deep Brain Stimulation (CL-DBS) is a therapy that can alleviate the symptoms of PD. The CL-DBS system consists of an electrode sending electrical stimulation signals to a specific region of the brain and a battery-powered stimulator implanted in the chest. The electrical stimuli in CL-DBS systems need to be adjusted in real-time in accordance with the state of PD symptoms. Therefore, fast and precise monitoring of PD symptoms is a critical function for CL-DBS systems. However, the current CL-DBS techniques suffer from high computational demands for real-time PD symptom monitoring, which are not feasible for implanted and wearable medical devices.MethodsIn this paper, we present an energy-efficient neuromorphic PD symptom detector using memristive three-dimensional integrated circuits (3D-ICs). The excessive oscillation at beta frequencies (13–35 Hz) at the subthalamic nucleus (STN) is used as a biomarker of PD symptoms.ResultsSimulation results demonstrate that our neuromorphic PD detector, implemented with an 8-layer spiking Long Short-Term Memory (S-LSTM), excels in recognizing PD symptoms, achieving a training accuracy of 99.74% and a validation accuracy of 99.52% for a 75%–25% data split. Furthermore, we evaluated the improvement of our neuromorphic CL-DBS detector using NeuroSIM. The chip area, latency, energy, and power consumption of our CL-DBS detector were reduced by 47.4%, 66.63%, 65.6%, and 67.5%, respectively, for monolithic 3D-ICs. Similarly, for heterogeneous 3D-ICs, employing memristive synapses to replace traditional Static Random Access Memory (SRAM) resulted in reductions of 44.8%, 64.75%, 65.28%, and 67.7% in chip area, latency, and power usage.DiscussionThis study introduces a novel approach for PD symptom evaluation by directly utilizing spiking signals from neural activities in the time domain. This method significantly reduces the time and energy required for signal conversion compared to traditional frequency domain approaches. The study pioneers the use of neuromorphic computing and memristors in designing CL-DBS systems, surpassing SRAM-based designs in chip design area, latency, and energy efficiency. Lastly, the proposed neuromorphic PD detector demonstrates high resilience to timing variations in brain neural signals, as confirmed by robustness analysis.

show abstract

BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

Kheradpisheh,

Mirsadeghi,

Masquelier

2020

Preprint

View full text Add to dashboard Cite

We recently proposed the S4NN algorithm, essentially an adaptation of backpropagation to multilayer spiking neural networks that use simple nonleaky integrate-and-fire neurons and a form of temporal coding known as time-to-first-spike coding. With this coding scheme, neurons fire at most once per stimulus, but the firing order carries information. Here, we introduce BS4NN, a modification of S4NN in which the synaptic weights are constrained to be binary (+1 or -1), in order to decrease memory and computation footprints. This was done using two sets of weights: firstly, real-valued weights, updated by gradient descent, and used in the backward pass of backpropagation, and secondly, their signs, used in the forward pass. Similar strategies have been used to train (non-spiking) binarized neural networks. The main difference is that BS4NN operates in the time domain: spikes are propagated sequentially, and different neurons may reach their threshold at different times, which increases computational power. We validated BS4NN on two popular benchmarks, MNIST and Fashion MNIST, and obtained state-of-the-art accuracies for this sort of networks (97.0% and 87.3% respectively) with a negligible accuracy drop with re- *

show abstract

A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design

Cited by 3 publications

References 48 publications

Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks

Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks

Monitoring time domain characteristics of Parkinson’s disease using 3D memristive neuromorphic system

BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

Contact Info

Product

Resources

About