Heterogeneous FPGA-Based Cost-Optimal Design for Timing-Constrained CNNs

Jiang, Weiwen; Sha, Edwin Hsing-Mean; Zhuge, Qingfeng; Yang, Lei; Chen, Xianzhang; Hu, Jingtong

doi:10.1109/tcad.2018.2857098

Cited by 37 publications

(23 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To fully utilize the computation power provided by multiple FPGAs, a typical technique is to implement the neural network on multiple FPGAs in a pipelined fashion [15], [20], [22], [23]. Figure 2 demonstrates one such example, in which a 5-layer network is partitioned into 3 pipeline stages, and each pipeline stage is mapped to a certain FPGA in an available pool.…”

Section: B Implementing Dnns On Fpgasmentioning

confidence: 99%

“…Our co-exploration concept and the general framework, however, can also be easily extended to other hardware platforms such as ASICs. Since timing performance on a single FPGA is limited by its restricted resource, it is prevalent to organize multiple FPGAs in a pipelined fashion [20]- [23] to provide high throughput (frame per second, FPS). In such a system, the pipeline efficiency is one of the most important metrics needing to be maximized, since it determines the hardware utilization as well as energy efficiency.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hardware/Software Co-Exploration of Neural Architectures

Jiang

Yang

Sha

et al. 2020

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

Self Cite

116

View full text Add to dashboard Cite

We propose a novel hardware and software coexploration framework for efficient neural architecture search (NAS). Different from existing hardware-aware NAS which assumes a fixed hardware design and explores the neural architecture search space only, our framework simultaneously explores both the architecture search space and the hardware design space to identify the best neural architecture and hardware pairs that maximize both test accuracy and hardware efficiency. Such a practice greatly opens up the design freedom and pushes forward the Pareto frontier between hardware efficiency and test accuracy for better design tradeoffs. The framework iteratively performs a two-level (fast and slow) exploration. Without lengthy training, the fast exploration can effectively fine-tune hyperparameters and prune inferior architectures in terms of hardware specifications, which significantly accelerates the NAS process. Then, the slow exploration trains candidates on a validation set and updates a controller using the reinforcement learning to maximize the expected accuracy together with the hardware efficiency. Experiments on ImageNet show that our co-exploration NAS can find the neural architectures and associated hardware design with the same accuracy, 35.24% higher throughput, 54.05% higher energy efficiency and 136× reduced search time, compared with the state-of-the-art hardware-aware NAS.

show abstract

Section: B Implementing Dnns On Fpgasmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hardware/Software Co-Exploration of Neural Architectures

Jiang

Yang

Sha

et al. 2020

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

Self Cite

116

View full text Add to dashboard Cite

show abstract

“…In [16], the authors focus on designing optimal pipelined CNNs on a set of heterogeneous FPGAs. The rationale is that different tasks in the pipeline are better suited to a specific type of FPGA.…”

Section: Related Workmentioning

confidence: 99%

CNN-on-AWS: Efficient Allocation of Multikernel Applications on Multi-FPGA Platforms

Shan

Lazarescu

Cortadella

et al. 2021

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Multi-FPGA platforms, like Amazon AWS F1, can run in the cloud multi-kernel pipelined applications, like Convolutional Neural Networks (CNNs), with excellent performance and lower energy consumption than CPUs or GPUs. We propose a method to efficiently map these applications on multi-FPGA platforms to maximize the application throughput. Our methodology finds, for the given resources, the optimal number of parallel instances of each kernel in the pipeline and their allocation to one or more among the available FPGAs. We obtain this by formulating and solving a mixed-integer, nonlinear optimization problem, in which we model the performance of each component and the duration of the phases in which the accelerated computation can be split into, namely: 1) data transfer from a host CPU to the DDR memory of each FPGA, 2) data transfer from FPGA DDR to FPGA on-chip memory, 3) kernel computation on the FPGA, 4) data transfer from FPGA on-chip memory to FPGA DDR, 5) data transfer from FPGA DDR to host. Finding the optimal solution using a Mixed-Integer Non-Linear Programming (MINLP) solver is often highly inefficient. Hence, we provide a fast heuristic method that according to our experiments can be much more efficient than the MINLP solver and finds comparable results. For larger problems (more CNN layers), our heuristic method can quickly find (several thousand times faster) much better solutions than the MINLP solver, even if we run the latter for a very long time.

show abstract

“…Most recently, with the growing demand in time performance, it is a trend to employ a cluster of FPGAs to execute DNNs [15,[26][27][28][29][30][31][32]. In [15,28], authors construct multiple FPGAs as a pipeline to execute a set of input images in a pipeline fashion.…”

Section: Related Workmentioning

confidence: 99%

“…In [15,28], authors construct multiple FPGAs as a pipeline to execute a set of input images in a pipeline fashion. In [26], authors split the CNN layers to balance pipeline stages for higher throughput and lower cost. Authors in [27] employ multiple FPGAs for the training phase.…”

Section: Related Workmentioning

confidence: 99%

Achieving Super-Linear Speedup across Multi-FPGA for Real-Time DNN Inference

Jiang

Sha

Zhang

et al. 2019

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

Real-time Deep Neural Network (DNN) inference with low-latency requirement has become increasingly important for numerous applications in both cloud computing (e.g., Apple's Siri) and edge computing (e.g., Google/Waymo's driverless car). FPGA-based DNN accelerators have demonstrated both superior flexibility and performance; in addition, for real-time inference with low batch size, FPGA is expected to achieve further performance improvement. However, the performance gain from the single-FPGA design is obstructed by the limited on-chip resource. In this paper, we employ multiple FPGAs to cooperatively run DNNs with the objective of achieving super-linear speed-up against single-FPGA design. In implementing such systems, we found two barriers that hinder us from achieving the design goal: (1) the lack of a clear partition scheme for each DNN layer to fully exploit parallelism, and (2) the insufficient bandwidth between the off-chip memory and the accelerator due to the growing size of DNNs. To tackle these issues, we propose a general framework, "Super-LIP", which can support different kinds of DNNs. In this paper, we take Convolutional Neural Network (CNN) as a vehicle to illustrate Super-LIP. We first formulate an accurate system-level model to support the exploration of best partition schemes. Then, we develop a novel design methodology to effectively alleviate the heavy loads on memory bandwidth by moving traffic from memory bus to inter-FPGA links. We implement Super-LIP based on ZCU102 FPGA boards. Results demonstrate that Super-LIP with 2 FPGAs can achieve 3.48× speedup, compared to the state-of-the-art single-FPGA design. What is more, as the number of FPGAs scales up, the system latency can be further reduced while maintaining high energy efficiency.

show abstract

Heterogeneous FPGA-Based Cost-Optimal Design for Timing-Constrained CNNs

Cited by 37 publications

References 18 publications

Hardware/Software Co-Exploration of Neural Architectures

Hardware/Software Co-Exploration of Neural Architectures

CNN-on-AWS: Efficient Allocation of Multikernel Applications on Multi-FPGA Platforms

Achieving Super-Linear Speedup across Multi-FPGA for Real-Time DNN Inference

Contact Info

Product

Resources

About