An FPGA-based MobileNet Accelerator Considering Network Structure Characteristics

Shi, Yan; Liu, Zhengyan; Wang, Yun; Zeng, Chenglong; Liu, Qiang; Cheng, Bowen; Cheung, Ray C. C.

doi:10.1109/fpl53798.2021.00011

Cited by 22 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each PE undertakes a convolution computation, limiting the parallelism on sliding input windows' dimensions, such as channel dimension or vector dimension. Previous works [7], [8] designed configurable adder trees to support different convolutions and exploit the inherent parallelism in convolutions. The reconfigurable adder tree can be reconfigured to process x additions, each adding y input data simultaneously, making it compatible with the parallelism in various convolutions.…”

Section: A Architecture Overviewmentioning

confidence: 99%

“…They have high computational complexity and tremendous parameters, leading to a large memory footprint and power budget and are difficult to be deployed on resource-constrained platforms. DNN compression methods [4] and dedicated, efficient hardware accelerators [5], [6], [7], [8] were explored to deal with this problem. Network compression methods include data quantization [9], [10], sparsity exploration [11], [12], and compact model design [13], [14].…”

Section: Introductionmentioning

confidence: 99%

“…MobileNet [13], [15] as a typical representative of compact models, adopts depthwise separable convolution (DSC), which can achieve a comparable accuracy with much fewer multiply-and-accumulation (MAC) operations and parameters. To further improve throughput and energy efficiency, dedicated hardware accelerators for compact models [7], [8] were designed to explore computing parallelism and efficient memory organization.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

Chen

Gao

2022

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are promising 1 alternatives to artificial neural networks (ANNs) since they 2 are more realistic brain-inspired computing models. SNNs have 3 sparse neuron firing over time, i.e., spatiotemporal sparsity; thus, 4 they are helpful in enabling energy-efficient hardware inference. 5 However, exploiting the spatiotemporal sparsity of SNNs in 6 hardware leads to unpredictable and unbalanced workloads, 7 degrading the energy efficiency. Compared to SNNs with sim-8 ple fully connected structures, those extensive structures (e.g., 9 standard convolutions, depthwise convolutions, and pointwise 10 convolutions) can deal with more complicated tasks but lead 11 to difficulties in hardware mapping. In this work, we propose 12 a novel reconfigurable architecture, Cerebron, which can fully 13 exploit the spatiotemporal sparsity in SNNs with maximized 14 data reuse and propose optimization techniques to improve the 15 efficiency and flexibility of the hardware. To achieve flexibility, 16 the reconfigurable compute engine is compatible with a variety of 17 spiking layers and supports inter-computing-unit (CU) and intra-18 CU reconfiguration. The compute engine can exploit data reuse 19 and guarantee parallel data access when processing different 20 convolutions to achieve memory efficiency. A two-step data 21 sparsity exploitation method is introduced to leverage the sparsity 22 of discrete spikes and reduce the computation time. Besides, 23 an online channelwise workload scheduling strategy is designed 24 to reduce the latency further. Cerebron is verified on image 25 segmentation and classification tasks using a variety of state-of-26 the-art spiking network structures. Experimental results show 27 that Cerebron has achieved at least 17.5× prediction energy 28 reduction and 20× speedup compared with state-of-the-art field-29 programmable gate array (FPGA)-based accelerators.30

show abstract

Section: A Architecture Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

Chen

Gao

2022

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

show abstract

“…Prior works have shown that FPGAs offer attractive performance and power efficiency for DNN inference applications [1], [2], [3], [4]. For mapping a DNN model to an FPGA, the quantization of weights and activations is the key optimization.…”

Section: Introductionmentioning

confidence: 99%

An Exploration of State-of-the-Art Automation Frameworks for FPGA-Based DNN Acceleration

et al. 2023

View full text Add to dashboard Cite

FPGA-based acceleration is considered a promising approach to improve the performance and power efficiency of Deep Neural Network (DNN) inference tasks. However, mapping a DNN onto an FPGA is not trivial. To make this easier, various automation frameworks have been proposed. Among them, FINN and Vitis AI, both developed by Xilinx, are two key players. They represent two different philosophies in designing FPGA-based DNN accelerators: dataflow-style and overlay-style architectures. Dataflow architectures are generally expected to provide better performance and power efficiency but have a major drawback in that they scale very poorly to the size of the target DNN. Advanced frameworks like FINN alleviate this drawback by transforming the target DNN into operations that can be time-multiplexed on fewer hardware resources. This approach, however, is challenging because of the difficulty in transforming the target DNN and raises a question as to whether the generated dataflow architectures retain a significant advantage over overlay architectures in terms of performance and power efficiency. This paper aims to clarify it by conducting an in-depth exploration of FINN and Vitis AI. For this purpose, we extend the FINN's development flow to be able to use the same target hardware and DNN model to evaluate each framework. We demonstrate the effectiveness of the FPGA-based acceleration by providing a comparison with two reference platforms: an NVIDIA Jetson Nano Developer Kit with a similar power budget to our target FPGA hardware, and a high-performance desktop computer with an Intel Core i7-11700K CPU. The results show that despite the use of the time-multiplexing approach, the FINN-based accelerator can still outperform the Vitis-AI-based accelerator by a significant margin, 8.4x in terms of latency, 3.0x in terms of throughput, and 3.3x in terms of power efficiency. The outcome of the comparison with the NVIDIA Jetson Nano Developer Kit and the desktop computer is also overwhelmingly favorable to the FINN-based accelerator. This indicates that, even in the case of using automation frameworks, DNN accelerators on FPGAs can still yield significant performance and power efficiency gains compared with GPUs and CPUs.

show abstract

A real-time and high-performance MobileNet accelerator based on adaptive dataflow scheduling for image classification

Sang,

Ruan,

et al. 2023

J Real-Time Image Proc

View full text Add to dashboard Cite

Convolutional neural network (CNN) models equipped with depth separable convolution (DSC) promise a lower spatial complexity while retaining high model accuracy. However, little attention has been paid to their hardware architecture. Previous studies on DSC-based CNN accelerators typically use fixed computational models for various models, leading to an imbalance between power, efficiency, and performance. To address this problem, a novel, real-time DSCbased CNN accelerator that can accommodate fieldprogrammable gate arrays (FPGAs) of different capacities and CNNs of different sizes is proposed in this paper. Attractively, a dynamically reconfigurable computing engine and blockconvolution-based adaptive dataflow scheduling mode strike a trade-off between hardware resources and the processing speed in industrial processes. The proposed MobileNet accelerator was implemented and evaluated on the Xilinx XC7020 platform. Compared to previous FPGA-based accelerators, the experimental results showed that our approach can provide a frame rate of 29.4 FPS for full HD RGB images, meeting the needs of real industrial real-time applications.

show abstract

An FPGA-based MobileNet Accelerator Considering Network Structure Characteristics

Cited by 22 publications

References 15 publications

Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

An Exploration of State-of-the-Art Automation Frameworks for FPGA-Based DNN Acceleration

A real-time and high-performance MobileNet accelerator based on adaptive dataflow scheduling for image classification

Contact Info

Product

Resources

About