A Programmable and Flexible Vision Processor

Luo, Qian; Yao, Chunhe; Ning, Ke; Zheng, Xuemin; Zhao, Mingxin; Cheng, Li; Yu, Shuangming; Liu, Jian; Wu, Nanjian; Liu, Liyuan

doi:10.1109/tcsii.2022.3181161

Cited by 6 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To ensure a fair comparison with several previously state-of-the-art results, we designed the networks to have the same or similar size. As we focused on edge applications, we limited the network weights to a few megabytes (Luo et al, 2022 ; Wang et al, 2022 ).…”

Section: Experiments and Resultsmentioning

confidence: 99%

DT-SCNN: dual-threshold spiking convolutional neural network with fewer operations and memory access for edge applications

Lei,

Yang,

Liu

et al. 2024

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

The spiking convolutional neural network (SCNN) is a kind of spiking neural network (SNN) with high accuracy for visual tasks and power efficiency on neuromorphic hardware, which is attractive for edge applications. However, it is challenging to implement SCNNs on resource-constrained edge devices because of the large number of convolutional operations and membrane potential (Vm) storage needed. Previous works have focused on timestep reduction, network pruning, and network quantization to realize SCNN implementation on edge devices. However, they overlooked similarities between spiking feature maps (SFmaps), which contain significant redundancy and cause unnecessary computation and storage. This work proposes a dual-threshold spiking convolutional neural network (DT-SCNN) to decrease the number of operations and memory access by utilizing similarities between SFmaps. The DT-SCNN employs dual firing thresholds to derive two similar SFmaps from one Vm map, reducing the number of convolutional operations and decreasing the volume of Vms and convolutional weights by half. We propose a variant spatio-temporal back propagation (STBP) training method with a two-stage strategy to train DT-SCNNs to decrease the inference timestep to 1. The experimental results show that the dual-thresholds mechanism achieves a 50% reduction in operations and data storage for the convolutional layers compared to conventional SCNNs while achieving not more than a 0.4% accuracy loss on the CIFAR10, MNIST, and Fashion MNIST datasets. Due to the lightweight network and single timestep inference, the DT-SCNN has the least number of operations compared to previous works, paving the way for low-latency and power-efficient edge applications.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

DT-SCNN: dual-threshold spiking convolutional neural network with fewer operations and memory access for edge applications

Lei,

Yang,

Liu

et al. 2024

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The im2col process consumes more clock cycles computing depthwise Conv because the vanilla TPU-like array is not optimized for single-channel Conv. We also compared GEMM between the accelerator and a typical 2D-Mapping processor [1], which supports SIMD fashion vector computation. When performing a GEMM of a 16 × 1024 matrix and a 1024 × 1024 matrix, our accelerator achieves 85.7% latency improvement.…”

Section: Discussionmentioning

confidence: 99%

“…As Figure 1 shows, current accelerators are divided into 2D mapping and systolic architectures. 2D mapping accelerators [1] implement finegrained operations through SIMD, which provides strong programmability, but their throughput is limited by memory bandwidth and complex broadcast networks. Systolic architecture accelerators [2][3][4] achieve high throughput but has lower flexibility.…”

mentioning

confidence: 99%

A Conv‐GEMM reconfigurable accelerator with WS‐RS dataflow for high throughput processing

Wang,

Zhang,

Deng

et al. 2024

Electronics Letters

View full text Add to dashboard Cite

Convolution and matrix operations are both important computations in Deep Neural Networks (DNNs). However, the significant differences between convolution and matrix computation patterns have posed a challenge in efficiently supporting both convolution (Conv) and general matrix multiplication (GEMM) on hardware design. This paper proposes a Conv‐GEMM reconfigurable accelerator architecture for high throughput edge processing. A weight stationary‐row streaming (WS‐RS) dataflow scheme is proposed, which maximizes data reuse through hierarchical memory structures and flexible PE connections, and supports high throughput edge‐based deep learning algorithms. Based on the proposed dataflow, multi‐scale memory access network (MMAN), reconfigurable accumulator array (RAA), and configurable instruction set architecture (ISA) are designed to optimize computation throughput and energy efficiency. The accelerator is designed under 65 nm technology, achieves peak performance of 1.15 TOPS at 250 MHz, with an energy efficiency of 1.14 TOPS/W. The GEMM computation achieves 85.7% latency improvement and the Mobilenet‐V1 processing achieves a throughput of 529 fps under a 256 × 224 image size and an 87.15% (top‐5) accuracy on the ImageNet dataset.

show abstract

Floorplanning and Power/Ground Network Design for A Programmable Vision Chip

Wei

et al. 2022

2022 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA)

View full text Add to dashboard Cite

A Programmable and Flexible Vision Processor

Cited by 6 publications

References 17 publications

DT-SCNN: dual-threshold spiking convolutional neural network with fewer operations and memory access for edge applications

DT-SCNN: dual-threshold spiking convolutional neural network with fewer operations and memory access for edge applications

A Conv‐GEMM reconfigurable accelerator with WS‐RS dataflow for high throughput processing

Floorplanning and Power/Ground Network Design for A Programmable Vision Chip

Contact Info

Product

Resources

About