Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA

Shen, Junzhong; Huang, Yawen; Wang, Zelong; Qiao, Yan; Wen, Mei; Zhang, Chunyuan

doi:10.1145/3174243.3174257

Cited by 77 publications

(45 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As shown in [17], the Winograd algorithm reduces the computation complexity significantly in CNNs. However, there are still some limitations in terms of flexibility.…”

Section: Resultsmentioning

confidence: 99%

“…The additions are executed with LUTs instead of DSP slices. That is why the implementation in [17] achieves the best performance density among all the listed implementations. In terms of throughput, our accelerator achieves state-of-the-art performance on both VGG and C3D.…”

Section: Methodsmentioning

confidence: 95%

“…As shown in Table 5, our accelerator is better than [18,22], and within 5% of [19] in terms of performance density. The implementation in [17] adopts the Winograd algorithm, which reduces the multiplications in 3D convolutional layers by 70.4% at the cost of 101% more additions. The additions are executed with LUTs instead of DSP slices.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

A Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs

Liu

Chow

et al. 2019

Electronics

View full text Add to dashboard Cite

Three-dimensional convolutional neural networks (3D CNNs) have gained popularity in many complicated computer vision applications. Many customized accelerators based on FPGAs are proposed for 2D CNNs, while very few are for 3D CNNs. Three-D CNNs are far more computationally intensive and the design space for 3D CNN acceleration has been further expanded since one more dimension is introduced, making it a big challenge to accelerate 3D CNNs on FPGAs. Motivated by the finding that the computation patterns of 2D and 3D CNNs are very similar, we propose a uniform architecture design for accelerating both 2D and 3D CNNs in this paper. The uniform architecture is based on the idea of mapping convolutions to matrix multiplications. A customized mapping module is developed to generate the feature matrix tilings with no need to store the entire enlarged feature matrix on-chip or off-chip, a splitting strategy is adopted to reconstruct a convolutional layer to adapt to the on-chip memory capacity, and a 2D multiply-and-accumulate (MAC) array is adopted to compute matrix multiplications efficiently. For demonstration, we implement an accelerator prototype with a high-level synthesis (HLS) methodology on a Xilinx VC709 board and test the accelerator on three typical CNN models: AlexNet, VGG16, and C3D. Experimental results show that the accelerator achieves state-of-the-art throughput performance on both 2D and 3D CNNs, with much better energy efficiency than the CPU and GPU.

show abstract

“…As shown in [17], the Winograd algorithm reduces the computation complexity significantly in CNNs. However, there are still some limitations in terms of flexibility.…”

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 95%

See 1 more Smart Citation

A Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs

Liu

Chow

et al. 2019

Electronics

View full text Add to dashboard Cite

show abstract

“…Many existing frameworks [9], [25], [31], [23], [33], [39], that map CNN models to FPGAs generate a large homogeneous processing core that is temporally shared among layers. This common design is flexible, as by sequentially carrying out convolutions, it is less constrained by the amount of resources available on FPGAs.…”

Section: Related Workmentioning

confidence: 99%

Automatic Generation of Multi-Precision Multi-Arithmetic CNN Accelerators for FPGAs

Zhao

Gao

Guo

et al. 2019

2019 International Conference on Field-Programmable Technology (ICFPT)

View full text Add to dashboard Cite

Modern deep Convolutional Neural Networks (CNNs) are computationally demanding, yet real applications often require high throughput and low latency. To help tackle these problems, we propose Tomato, a framework designed to automate the process of generating efficient CNN accelerators. The generated design is pipelined and each convolution layer uses different arithmetics at various precisions. Using Tomato, we showcase state-of-the-art multi-precision multi-arithmetic networks, including MobileNet-V1, running on FPGAs. To our knowledge, this is the first multi-precision multi-arithmetic autogeneration framework for CNNs. In software, Tomato fine-tunes pretrained networks to use a mixture of short powers-of-2 and fixed-point weights with a minimal loss in classification accuracy. The fine-tuned parameters are combined with the templated hardware designs to automatically produce efficient inference circuits in FPGAs. We demonstrate how our approach significantly reduces model sizes and computation complexities, and permits us to pack a complete ImageNet network onto a single FPGA without accessing off-chip memories for the first time. Furthermore, we show how Tomato produces implementations of networks with various sizes running on single or multiple FPGAs. To the best of our knowledge, our automatically generated accelerators outperform closest FPGA-based competitors by at least 2-4× for lantency and throughput; the generated accelerator runs ImageNet classification at a rate of more than 3000 frames per second.

show abstract

“…Due to the recent popularity of 2D CNNs in image recognition and related tasks, there have been a plethora of works proposing new architectures to accelerate 2D CNNs [8], [10], [11], [49]. There have been no works that accelerate 3D CNNs in ASICs, although we note several recent works [50], [51] which have explored hardware acceleration of 3D CNNs on FPGAs. On one hand, FPGAs are reconfigurable and thus can directly adapt to different CNN configurations.…”

Section: F Hardware Implementationmentioning

confidence: 99%

Morph: Flexible Acceleration for 3D CNN-Based Video Understanding

Hegde

Agrawal

Yao

et al. 2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition has received far less attention as an accelerator target. This is surprising, as video recognition is more computationally intensive than image recognition, and video traffic is predicted to be the majority of internet traffic in the coming years.This paper fills the gap between algorithmic and hardware advances for video recognition by providing a design space exploration and flexible architecture for accelerating 3D Convolutional Neural Networks (3D CNNs)-the core kernel in modern video understanding. When compared to (2D) CNNs used for image recognition, efficiently accelerating 3D CNNs poses a significant engineering challenge due to their large (and variable over time) memory footprint and higher dimensionality.To address these challenges, we design a novel accelerator, called Morph, that can adaptively support different spatial and temporal tiling strategies depending on the needs of each layer of each target 3D CNN. We codesign a software infrastructure alongside the Morph hardware to find good-fit parameters to control the hardware. Evaluated on state-of-the-art 3D CNNs, Morph achieves up to 3.4× (2.5× average) reduction in energy consumption and improves performance/watt by up to 5.1× (4× average) compared to a baseline 3D CNN accelerator, with an area overhead of 5%. Morph further achieves a 15.9× average energy reduction on 3D CNNs when compared to Eyeriss.

show abstract

Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA

Cited by 77 publications

References 18 publications

A Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs

A Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs

Automatic Generation of Multi-Precision Multi-Arithmetic CNN Accelerators for FPGAs

Morph: Flexible Acceleration for 3D CNN-Based Video Understanding

Contact Info

Product

Resources

About