Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC

Nurvitadhi, Eriko; Sim, Jaewoong; Sheffield, David; Mishra, Asit K.; Krishnan, Srivatsan; Marr, Debbie

doi:10.1109/fpl.2016.7577314

Cited by 138 publications

(59 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GRU on FPGA Nurvitadhi et al presented a hardware accelerator for Gated Recurrent Network (GRU) on Stratix V and Arria 10 FPGAs [16]. This work shows that FPGA can provide superior performance/Watt over CPU and GPU.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ese

Han

Kang²,

Mao

et al. 2017

Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

458

View full text Add to dashboard Cite

Long Short-Term Memory (LSTM) is widely used in speech recognition. In order to achieve higher prediction accuracy, machine learning scientists have built increasingly larger models. Such large models are both computation and memory intensive. Deploying such bulky models results in high power consumption and leads to a high total cost of ownership (TCO) for a data center.To speedup prediction and make it energy efficient, we first propose a load-balance-aware pruning method that can compress the LSTM model size by 20× (10× from pruning and 2× from quantization) with negligible loss of prediction accuracy. Also we proposed load-balance-aware pruning to ensure high hardware utilization. Next, we propose a scheduler that encodes and partitions the compressed model to multiple PEs for parallelism and schedules the complicated LSTM data flow. Finally, we design a hardware architecture named ESE that works directly on the sparse LSTM model.Implemented on Xilinx XCKU060 FPGA running at 200MHz, ESE has a performance of 282 GOPS working directly on the sparse LSTM network, corresponding to 2.52 TOPS on the dense one, and processes a full LSTM for speech recognition with a power dissipation of 41 Watts. Evaluated on the LSTM for speech recognition benchmark, ESE is 43× and 3× faster than Core i7 5930k CPU and Pascal Titan X GPU implementations. It achieves 40× and 11.5× higher energy efficiency compared with the CPU and GPU respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

“…We adjust the cache depth to investigate its effect. The FIFO width is 16-bit, and its depth is set at 1,4,8,16. In Fig.13, when the FIFO depth is one (no FIFO), the utilization, which is defined as busy cycle divided by total cycles, is low (80%) due to load imbalance.…”

Section: Resource Utilizationmentioning

confidence: 99%

Ese

Han

Kang²,

Mao

et al. 2017

Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

458

View full text Add to dashboard Cite

show abstract

“…Recently, heterogeneous hardware platforms present the potential to accelerate specific deep learning algorithms while reducing the processing time and energy consumption [53], [54]. For the hardware on EI, various heterogeneous hardware are developed for particular EI application scenario to address the resource limitation problem in the edge.…”

Section: Hardwarementioning

confidence: 99%

OpenEI: An Open Framework for Edge Intelligence

Zhang

Wang

et al. 2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

100

View full text Add to dashboard Cite

1 Abstract-In the last five years, edge computing has attracted tremendous attention from industry and academia due to its promise to reduce latency, save bandwidth, improve availability, and protect data privacy to keep data secure. At the same time, we have witnessed the proliferation of AI algorithms and models which accelerate the successful deployment of intelligence mainly in cloud services. These two trends, combined together, have created a new horizon: Edge Intelligence (EI). The development of EI requires much attention from both the computer systems research community and the AI community to meet these demands.However, existing computing techniques used in the cloud are not applicable to edge computing directly due to the diversity of computing sources and the distribution of data sources. We envision that there missing a framework that can be rapidly deployed on edge and enable edge AI capabilities. To address this challenge, in this paper we first present the definition and a systematic review of EI. Then, we introduce an Open Framework for Edge Intelligence (OpenEI), which is a lightweight software platform to equip edges with intelligent processing and data sharing capability. We analyze four fundamental EI techniques which are used to build OpenEI and identify several open problems based on potential research directions. Finally, four typical application scenarios enabled by OpenEI are presented.

show abstract

“…Predictions have been made that as much as 30% of datacenter servers will have FPGAs by 2020 [6]. This suggests that FPGAs could become a common component in future servers and could play an important role as primary computing resources [23].…”

Section: Introductionmentioning

confidence: 99%

Automated Accelerator Generation and Optimization with Composable, Parallel and Pipeline Architecture

Cong

Wei

et al. 2018

2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)

View full text Add to dashboard Cite

CPU-FPGA heterogeneous architectures are attracting ever-increasing attention in an attempt to advance computational capabilities and energy efficiency in today's datacenters. These architectures provide programmers with the ability to reprogram the FPGAs for flexible acceleration of many workloads. Nonetheless, this advantage is often overshadowed by the poor programmability of FPGAs whose programming is conventionally a RTL design practice. Although recent advances in high-level synthesis (HLS) significantly improve the FPGA programmability, it still leaves programmers facing the challenge of identifying the optimal design configuration in a tremendous design space.This paper aims to address this challenge and pave the path from software programs towards high-quality FPGA accelerators. Specifically, we first propose the composable, parallel and pipeline (CPP) microarchitecture as a template of accelerator designs. Such a well-defined template is able to support efficient accelerator designs for a broad class of computation kernels, and more importantly, drastically reduce the design space. Also, we introduce an analytical model to capture the performance and resource trade-offs among different design configurations of the CPP microarchitecture, which lays the foundation for fast design space exploration. On top of the CPP microarchitecture and its analytical model, we develop the AutoAccel framework to make the entire accelerator generation automated. AutoAccel accepts a software program as an input and performs a series of code transformations based on the result of the analytical-model-based design space exploration to construct the desired CPP microarchitecture. Our experiments show that the AutoAccel-generated accelerators outperform their corresponding software implementations by an average of 72x for a broad class of computation kernels.

show abstract

Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC

Cited by 138 publications

References 5 publications

Ese

Ese

OpenEI: An Open Framework for Edge Intelligence

Automated Accelerator Generation and Optimization with Composable, Parallel and Pipeline Architecture

Contact Info

Product

Resources

About