Optimization of Spiking Neural Networks Based on Binary Streamed Rate Coding

Al-Hamid, Ali A.; Kim, HyungWon

doi:10.3390/electronics9101599

Cited by 4 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 13 shows the case when the applied input image is ‘7’, the SNN predicts the correct classification result of ‘7’ by producing the maximum spiking activity at the 7th output node. The average classification accuracy achieved for the SNN is 94.66%, which is very close to the accuracy of 94.69% for the optimized SNN model in Python [ 41 ].…”

Section: Performance Analysismentioning

confidence: 53%

“…For the SNN implementation proposed in this paper, the SNN model for the MNIST dataset is first optimized using a low-cost spike signal representation technique called Binary Streamed Rate Coding (BSRC), which was presented in our previous work [ 41 ]. While the proposed SNN employs the off-chip training technique, it determines the SNN’s floating point weights by propagating sequences of spikes through an accurate model of the target SNN chip, instead of an ANN counterpart as in most other SNNs.…”

Section: Snn Implementation and Circuit Designmentioning

confidence: 99%

See 1 more Smart Citation

A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits

Asghar

Arslan

Kim

2021

Sensors

Self Cite

View full text Add to dashboard Cite

To realize a large-scale Spiking Neural Network (SNN) on hardware for mobile applications, area and power optimized electronic circuit design is critical. In this work, an area and power optimized hardware implementation of a large-scale SNN for real time IoT applications is presented. The analog Complementary Metal Oxide Semiconductor (CMOS) implementation incorporates neuron and synaptic circuits optimized for area and power consumption. The asynchronous neuronal circuits implemented benefit from higher energy efficiency and higher sensitivity. The proposed synapse circuit based on Binary Exponential Charge Injector (BECI) saves area and power consumption, and provides design scalability for higher resolutions. The SNN model implemented is optimized for 9 × 9 pixel input image and minimum bit-width weights that can satisfy target accuracy, occupies less area and power consumption. Moreover, the spiking neural network is replicated in full digital implementation for area and power comparisons. The SNN chip integrated from neuron and synapse circuits is capable of pattern recognition. The proposed SNN chip is fabricated using 180 nm CMOS process, which occupies a 3.6 mm2 chip core area, and achieves a classification accuracy of 94.66% for the MNIST dataset. The proposed SNN chip consumes an average power of 1.06 mW—20 times lower than the digital implementation.

show abstract

Section: Performance Analysismentioning

confidence: 53%

Section: Snn Implementation and Circuit Designmentioning

confidence: 99%

A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits

Asghar

Arslan

Kim

2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed SNN SoC architecture adopts a spike signal representation called Binary Streamed Rate Coding (BSRC), which has been introduced by [ 18 ]. It has been shown that BSRC allows for an optimized SNN model for compact hardware and employs direct training for high accuracy based on an off-chip training technique.…”

Section: Neural Network Architecturementioning

confidence: 99%

A Compact and Low-Power SoC Design for Spiking Neural Network Based on Current Multiplier Charge Injector Synapse

Asghar

Arslan²,

Al-Hamid

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

This paper presents a compact analog system-on-chip (SoC) implementation of a spiking neural network (SNN) for low-power Internet of Things (IoT) applications. The low-power implementation of an SNN SoC requires the optimization of not only the SNN model but also the architecture and circuit designs. In this work, the SNN has been constituted from the analog neuron and synaptic circuits, which are designed to optimize both the chip area and power consumption. The proposed synapse circuit is based on a current multiplier charge injector (CMCI) circuit, which can significantly reduce power consumption and chip area compared with the previous work while allowing for design scalability for higher resolutions. The proposed neuron circuit employs an asynchronous structure, which makes it highly sensitive to input synaptic currents and enables it to achieve higher energy efficiency. To compare the performance of the proposed SoC in its area and power consumption, we implemented a digital SoC for the same SNN model in FPGA. The proposed SNN chip, when trained using the MNIST dataset, achieves a classification accuracy of 96.56%. The presented SNN chip has been implemented using a 65 nm CMOS process for fabrication. The entire chip occupies 0.96 mm2 and consumes an average power of 530 μW, which is 200 times lower than its digital counterpart.

show abstract

“…[29] downsized SNN models by 73.78% with low-precision parameters at the cost of 1.04% test error increase. Other works showed potential of low-precision SNNs on various object recognition datasets [1,27,39].…”

Section: Spiking Neural Networkmentioning

confidence: 99%

“…For successful deployment to neuromorphic chips, this integration in deep SNNs is inevitable and more important than in ANNs. However, most of the studies, unlike those of ANNs, have focused on quantization [1,27,29,39] and pruning [20,30] in deep SNNs. This is because knowledge distillation indispensably requires training procedure, which is challenging in deep SNNs.…”

Section: Introductionmentioning

confidence: 99%

Energy-efficient Knowledge Distillation for Spiking Neural Networks

Park¹,

Kim²,

Doh³

et al. 2021

Preprint

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have been gaining interest as energy-efficient alternatives of conventional artificial neural networks (ANNs) due to their event-driven computation. Considering the future deployment of SNN models to constrained neuromorphic devices, many studies have applied techniques originally used for ANN model compression, such as network quantization, pruning, and knowledge distillation, to SNNs. Among them, existing works on knowledge distillation reported accuracy improvements of student SNN model. However, analysis on energy efficiency, which is also an important feature of SNN, was absent. In this paper, we thoroughly analyze the performance of the distilled SNN model in terms of accuracy and energy efficiency. In the process, we observe a substantial increase in the number of spikes, leading to energy inefficiency, when using the conventional knowledge distillation methods. Based on this analysis, to achieve energy efficiency, we propose a novel knowledge distillation method with heterogeneous temperature parameters. We evaluate our method on two different datasets and show that the resulting SNN student satisfies both accuracy improvement and reduction of the number of spikes. On MNIST dataset, our proposed student SNN achieves up to 0.09% higher accuracy and produces 65% less spikes compared to the student SNN trained with conventional knowledge distillation method. We also compare the results with other SNN compression techniques and training methods.

show abstract

Optimization of Spiking Neural Networks Based on Binary Streamed Rate Coding

Cited by 4 publications

References 27 publications

A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits

A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits

A Compact and Low-Power SoC Design for Spiking Neural Network Based on Current Multiplier Charge Injector Synapse

Energy-efficient Knowledge Distillation for Spiking Neural Networks

Contact Info

Product

Resources

About