SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks

Yin, Ruokai; Moitra, Abhishek; Bhattacharjee, Abhiroop; Kim, Youngeun; Panda, Priyadarshini

doi:10.48550/arxiv.2204.05422

“…Original SNNs Near-Memory Computing Analog-Digital-Mixed, Large-Scale BrainScaleS [87] SNNs, Learning Near-Memory Computing Analog-Digital-Mixed, Large-Scale SpiNNaker [81] SNNs, Learning Near-Memory Computing Digital, Large-Scale TrueNorth [14] SNNs Near-Memory Computing Digital, Large-Scale Darwin [83] SNNs Near-Memory Computing Digital, Small-Scale ROLLS [88] SNNs, Learning Near-Memory Computing Analog-Digital-Mixed, Small-Scale DYNAPs [94] SNNs Near-Memory Computing Analog-Digital-Mixed, Small-Scale Loihi [41] SNNs, Learning Near-Memory Computing Digital, Large-Scale Tianjic [85], [86] ANNs & SNNs Near-Memory Computing Digital, Large-Scale ODIN [89] SNNs, Learning Near-Memory Computing Digital, Small-Scale MorphIC [90] SNNs, Learning Near-Memory Computing Digital, Small-Scale DYNAPs-CNN/DYNAP-SE [44] SNNs Near-Memory Computing Digital, Small-Scale FlexLearn [99] SNNs, Learning ANN Accelerator Variants Digital, Large-Scale SpinalFlow [100] SNNs ANN Accelerator Variants Digital, Small-Scale H2Learn [101] SNNs, Learning ANN Accelerator Variants Digital, Large-Scale SATA [102] SNNs, Learning ANN Accelerator Variants Digital, Small-Scale BrainScaleS 2 [82] ANNs & SNNs, Learning Near-Memory Computing Digital, Large-Scale SpiNNaker 2 [95] ANNs & SNNs, Learning Near-Memory Computing Digital, Large-Scale Y. Kuang et al [282] ANNs & SNNs Near-Memory Computing Digital, Large-Scale SRAM/DRAM/Flash-based ANNs, SNNs In-Memory Computing Digital, Small-Scale Memristor-based ANNs, SNNs In-Memory Computing Analog-Digital-Mixed, Small-Scale neuromorphic workloads. The learning of SNNs on GPU is inefficient and hard to optimize [91].…”

Section: Chip Familymentioning

confidence: 99%

“…Recently, backpropagation through time (BPTT) learning has been applied to SNNs and demonstrated much higher accuracy compared to bio-plausible synaptic plasticity rules [67], [75]. Several works such as H2Learn [101] and SATA [102] design specific architectures for BPTT learning of SNNs. In the future, the incorporation of learning rules will be increasingly critical for BIC chips to explore large and complex neuromorphic models.…”

Section: Chip Familymentioning

confidence: 99%

“…This architecture can achieve significant acceleration since the spike activities of SNNs are quite sparse. SATA [102] combines multipliers and selectors in a PE array to allow the processing of both the high-precision matrix operation and the spiking matrix operation, which supports the matrix operations in both forward and backward passes.…”

Section: Chip Familymentioning

confidence: 99%

“…From the functionality perspective, existing BIC chips can be classified into three categories: supporting SNNs, or supporting both SNNs and ANNs [85], [86], and supporting learning rules [87]- [92]. From the architecture perspective, BIC chips belong to one of the following categories: near-memorycomputing architectures [14], [41], [44], [81]- [83], [85]- [90], [93]- [95] in-memory-computing architectures [96]- [98] and ANN accelerator variants [99]- [102]. From the implementation perspective, the trade-off becomes more complicated because many factors, including application scenarios, PPA (performance, power, and area), and programmability, should be comprehensively considered [103].…”

Section: Introduction Motivation and Overviewmentioning

confidence: 99%

See 2 more Smart Citations

Brain Inspired Computing: A Systematic Survey and Future Trends

Li¹,

Deng²,

Tang³

et al. 2023

Preprint

1

0

View full text Add to dashboard Cite

<p>Brain Inspired Computing (BIC) is an emerging research field that aims to build fundamental theories, models, hardware architectures, and application systems toward more general Artificial Intelligence (AI) by learning from the information processing mechanisms or structures/functions of biological nervous systems. It is regarded as one of the most promising research directions for future intelligent computing in the post-Moore era. In the past few years, various new schemes in this field have sprung up to explore more general AI. These works are quite divergent in the aspects of modeling/algorithm, software tool, hardware platform, and benchmark data, since BIC is an interdisciplinary field that consists of many different domains, including computational neuroscience, artificial intelligence, computer science, statistical physics, material science, microelectronics and so forth. This situation greatly impedes researchers from obtaining a clear picture and getting started in the right way. Hence, there is an urgent requirement to do a comprehensive survey in this field to help correctly recognize and analyze such bewildering methodologies. What are the key issues to enhance the development of BIC? What roles do the current mainstream technologies play in the general framework of BIC? Which techniques are truly useful in real-world applications? These questions largely remain open. To address the above issues, in this survey we first clarify the biggest challenge of BIC: how can AI models benefit from the recent advancements in computational neuroscience? With this challenge in mind, we will focus on discussing the concept of BIC and summarize four components of BIC infrastructure development: 1) modeling/algorithm; 2) hardware platform; 3) software tool; and 4) benchmark data. For each component, we will summarize its recent progress, main challenges to resolve, and future trends. On the basis of these studies, we present a general framework for the real-world applications of BIC systems, which is promising to benefit both AI and brain science. Finally, we claim that it is extremely important to build a research ecology to promote prosperity continuously in this field.</p>

show abstract

“…From the functionality perspective, existing BIC chips can be classified into three categories: supporting SNNs, or supporting both SNNs and ANNs [85], [86], and supporting learning rules [87]- [92]. From the architecture perspective, BIC chips belong to one of the following categories: near-memorycomputing architectures [14], [41], [44], [81]- [83], [85]- [90], [93]- [95] in-memory-computing architectures [96]- [98] and ANN accelerator variants [99]- [102]. From the implementation perspective, the trade-off becomes more complicated because many factors, including application scenarios, PPA (performance, power, and area), and programmability, should be comprehensively considered [103].…”

Section: Introduction Motivation and Overviewmentioning

confidence: 99%

Brain Inspired Computing: A Systematic Survey and Future Trends

Li¹,

Deng²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Brain Inspired Computing (BIC) is an emerging research field that aims to build fundamental theories, models, hardware architectures, and application systems toward more general Artificial Intelligence (AI) by learning from the information processing mechanisms or structures/functions of biological nervous systems. It is regarded as one of the most promising research directions for future intelligent computing in the post-Moore era. In the past few years, various new schemes in this field have sprung up to explore more general AI. These works are quite divergent in the aspects of modeling/algorithm, software tool, hardware platform, and benchmark data, since BIC is an interdisciplinary field that consists of many different domains, including computational neuroscience, artificial intelligence, computer science, statistical physics, material science, microelectronics and so forth. This situation greatly impedes researchers from obtaining a clear picture and getting started in the right way. Hence, there is an urgent requirement to do a comprehensive survey in this field to help correctly recognize and analyze such bewildering methodologies. What are the key issues to enhance the development of BIC? What roles do the current mainstream technologies play in the general framework of BIC? Which techniques are truly useful in real-world applications? These questions largely remain open. To address the above issues, in this survey we first clarify the biggest challenge of BIC: how can AI models benefit from the recent advancements in computational neuroscience? With this challenge in mind, we will focus on discussing the concept of BIC and summarize four components of BIC infrastructure development: 1) modeling/algorithm; 2) hardware platform; 3) software tool; and 4) benchmark data. For each component, we will summarize its recent progress, main challenges to resolve, and future trends. On the basis of these studies, we present a general framework for the real-world applications of BIC systems, which is promising to benefit both AI and brain science. Finally, we claim that it is extremely important to build a research ecology to promote prosperity continuously in this field.</p>

show abstract

Advancing energy efficiency of spiking neural network accelerator via dynamic predictive early stopping

Miao,

Ikeda

2024

IEICE Electron. Express

0

View full text Add to dashboard Cite

Spiking Neural Network (SNN) accelerators, recognized for their potential in neuromorphic processing, have yet to fully realize high energy efficiency, especially in widespread conventional non-event-based tasks, thus limiting their broader applicability. In response, we introduce an effective, cost-efficient method named dynamic predictive early stopping. This method enhances energy efficiency by predicting and stopping nearly 70% of non-essential computations during inference while ensuring nearlossless performance. The FPGA-based accelerator prototype demonstrated a 35% energy efficiency net improvement (to 208.74 GOPS/W, leading for its class). Simultaneously, the proposed method incorporates only minimal (<0.41%) extensions to hardware. With potential for further improvements and explorations, this method could bring substantial impact by being widely adopted by SNN accelerators.

show abstract

SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks

Cited by 4 publications

References 0 publications

Brain Inspired Computing: A Systematic Survey and Future Trends

Brain Inspired Computing: A Systematic Survey and Future Trends

Brain Inspired Computing: A Systematic Survey and Future Trends

Advancing energy efficiency of spiking neural network accelerator via dynamic predictive early stopping

Contact Info

Product

Resources

About