LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing

Wu, Zhenzhi; Zhang, Hehui; Lin, Yihan; Li, Guoqi; Wang, Meng; Tang, Ye

doi:10.1109/tnnls.2021.3073016

Cited by 44 publications

(22 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For ANNs, the two dimension convolutional neural network (2D-CNN) (Krizhevsky et al, 2012 ) has become a common tool for image classification. In order to train 2D-CNN on the ES-dataset, a common approach is to accumulate the events into event frames according to the time dimension and then reconstruct the gray images (Wu et al, 2020 ) for training. Here we use the Edge-Integral algorithm described in Figure 6 for reconstruction.…”

Section: Resultsmentioning

confidence: 99%

“…For SNNs, we choose an SNN based on leaky integrate-and-fire (LIF) neurons (Dayan and Abbott, 2001 ) and an SNN based on leaky integrate-and-analog-fire (LIAF) (Wu et al, 2020 ) neurons. Rate coding (Adrian and Zotterman, 1926 ) is used to decode the event information because the significance of the specific time when the spikes appear in this dataset is weaker than the number of spikes.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

Lin¹,

Ding²,

Qiang³

et al. 2021

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

With event-driven algorithms, especially spiking neural networks (SNNs), achieving continuous improvement in neuromorphic vision processing, a more challenging event-stream dataset is urgently needed. However, it is well-known that creating an ES-dataset is a time-consuming and costly task with neuromorphic cameras like dynamic vision sensors (DVS). In this work, we propose a fast and effective algorithm termed Omnidirectional Discrete Gradient (ODG) to convert the popular computer vision dataset ILSVRC2012 into its event-stream (ES) version, generating about 1,300,000 frame-based images into ES-samples in 1,000 categories. In this way, we propose an ES-dataset called ES-ImageNet, which is dozens of times larger than other neuromorphic classification datasets at present and completely generated by the software. The ODG algorithm implements image motion to generate local value changes with discrete gradient information in different directions, providing a low-cost and high-speed method for converting frame-based images into event streams, along with Edge-Integral to reconstruct the high-quality images from event streams. Furthermore, we analyze the statistics of ES-ImageNet in multiple ways, and a performance benchmark of the dataset is also provided using both famous deep neural network algorithms and spiking neural network algorithms. We believe that this work shall provide a new large-scale benchmark dataset for SNNs and neuromorphic vision.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

Lin¹,

Ding²,

Qiang³

et al. 2021

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In LIAF models [1], the spike activation function F will have continuous output, while the membrane reset signal is still binary.…”

Section: A Modelsmentioning

confidence: 99%

“…When the target data is static data (like images), we prefer to use pipe-S, while we need to train LIF-SNN on dynamic data (like videos), we use pipe-D. Both of the sub-pipeline can be divided into three steps: training the original ANN model, fine-tuning on the transitional models (a spike-activation ANN or a LIAF-SNN [1]) and fine-tuning on the LIF model. Warmup (WU) step and sharpened ReLU are adopted as the smoothing methods, and the detailed process is introduced in the Method section.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Rethinking Pretraining as a Bridge from ANNs to SNNs

Lin¹,

Hu²,

Ma³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are known as a typical kind of brain-inspired models with their unique features of rich neuronal dynamics, diverse coding schemes and low power consumption properties. How to obtain a high-accuracy model has always been the main challenge in the field of SNN. Currently, there are two mainstream methods, i.e., obtaining a converted SNN through converting a well-trained Artificial Neural Network (ANN) to its SNN counterpart or training an SNN directly. However, the inference time of a converted SNN is too long, while SNN training is generally very costly and inefficient. In this work, a new SNN training paradigm is proposed by combining the concepts of the two different training methods with the help of the pretrain technique and BP-based deep SNN training mechanism. We believe that the proposed paradigm is a more efficient pipeline for training SNNs. The pipeline includes pipe-S for static data transfer tasks and pipe-D for dynamic data transfer tasks. SOTA results are obtained in a large-scale event-driven dataset ES-ImageNet. For training acceleration, we achieve the same (or higher) best accuracy as similar LIF-SNNs using 1/10 training time on ImageNet-1K and 2/5 training time on ES-ImageNet and also provide a time-accuracy benchmark for a new dataset ES-UCF101. These experimental results reveal the similarity of the functions of parameters between ANNs and SNNs and also demonstrate the various potential applications of this SNN training pipeline.

show abstract

RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network

Yoo,

Lee,

Wang

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Neuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks (SNNs) that are expensive to train. In this work, a neural network architecture is proposed, reservoir nodes‐enabled neuromorphic vision sensing network (RN‐Net), based on dynamic temporal encoding by on‐sensor reservoirs and simple deep neural network (DNN) blocks. The reservoir nodes enable efficient temporal processing of asynchronous events by leveraging the native dynamics of the node devices, while the DNN blocks enable spatial feature processing. Combining these blocks in a hierarchical structure, the RN‐Net offers efficient processing for both local and global spatiotemporal features. RN‐Net executes dynamic vision tasks created by event‐based cameras at the highest accuracy reported to date at one order of magnitude smaller network size. The use of simple DNN and standard backpropagation‐based training rules further reduces implementation and training costs.

show abstract

LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing

Cited by 44 publications

References 53 publications

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

Rethinking Pretraining as a Bridge from ANNs to SNNs

RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network

Contact Info

Product

Resources

About