Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) are promising 1 alternatives to artificial neural networks (ANNs) since they 2 are more realistic brain-inspired computing models. SNNs have 3 sparse neuron firing over time, i.e., spatiotemporal sparsity; thus, 4 they are helpful in enabling energy-efficient hardware inference. 5 However, exploiting the spatiotemporal sparsity of SNNs in 6 hardware leads to unpredictable and unbalanced workloads, 7 degrading the energy efficiency. Compared to SNNs with sim-8 ple fully connected structures, those extensive structures (e.g., 9 standard convolutions, depthwise convolutions, and pointwise 10 convolutions) can deal with more complicated tasks but lead 11 to difficulties in hardware mapping. In this work, we propose 12 a novel reconfigurable architecture, Cerebron, which can fully 13 exploit the spatiotemporal sparsity in SNNs with maximized 14 data reuse and propose optimization techniques to improve the 15 efficiency and flexibility of the hardware. To achieve flexibility, 16 the reconfigurable compute engine is compatible with a variety of 17 spiking layers and supports inter-computing-unit (CU) and intra-18 CU reconfiguration. The compute engine can exploit data reuse 19 and guarantee parallel data access when processing different 20 convolutions to achieve memory efficiency. A two-step data 21 sparsity exploitation method is introduced to leverage the sparsity 22 of discrete spikes and reduce the computation time. Besides, 23 an online channelwise workload scheduling strategy is designed 24 to reduce the latency further. Cerebron is verified on image 25 segmentation and classification tasks using a variety of state-of-26 the-art spiking network structures. Experimental results show 27 that Cerebron has achieved at least 17.5× prediction energy 28 reduction and 20× speedup compared with state-of-the-art field-29 programmable gate array (FPGA)-based accelerators.30

FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks

“…The third type of neuromorphic hardware follows the scheme of the ANN accelerator design except for constructing dedicated hardware for synaptic operations and explores optimal dataflow for SNNs specifically [20]- [26]. These types of work require less area cost and achieve higher computing resource utilization.…”

“…where f is the system clock frequency, and M × N denotes the size of the systolic array. The peak GSOP/s calculation is the same as [20] and [24]. In FireFly, M denotes the number of columns in the systolic array, while N denotes the rows.…”

“…The built-in host CPU controller enables fast deployment of different SNN networks without the need to change the PL logic. In most Zynq-based SNN accelerators such as Cerebron [20], the host program in the Zynq processing system sends synaptic weights and binary input spike maps into the Zynq programmable logic and collects the output spike maps in different SNN layers. However, the control command sequence traveling between PS and PL through the low-performance AXI-Lite protocol induces nonnegligible latency, leaving the systolic array idle and reducing the average throughput.…”

Ultralow-power spiking neural networks for 1024-ary orbital angular momentum shift keying free-space optical communication

“…The spikes propagate through the network until reaching the output. We adopt the integrate-and-fire (IF) neuron model in the Spiking ORNN [47]. Suppose the IF neurons in layer l receive binary input spikes s l (t) at timestep t, the temporary membrane potential of neurons is updated according to the following equation:…”

“…With heavy investment from industry and government, e.g. IBM, Intel, and DARPA, SNNs are expected to be part of the future artificial intelligence (AI) portfolio [46,47].…”