Recent advances in efficient computation of deep convolutional neural networks

Cheng, Jian; Wang, Peisong; Li, Gang; Hu, Qinghao; Lu, Hanqing

doi:10.1631/fitee.1700789

Cited by 210 publications

(98 citation statements)

References 85 publications

(108 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) The Key Novelty of our Paper Related to NN Study: FPGAs are attractive devices to accelerate NN since they represent an intermediate point between the power and performance efficiency of ASICs and the programmability of CPUs and GPUs [67], [68], [69], [70]. One of the key components of FPGAs that directly impacts the performance of FPGAbased NNs is built-in BRAMs, due to the high-demand of NN computations for the parallel data access, as described in detail for recent FPGA-based accelerators in this survey paper [14].…”

Section: B Recent Related Studies On Nnsmentioning

confidence: 99%

Comprehensive Evaluation of Supply Voltage Underscaling in FPGA on-Chip Memories

Salami

Ünsal

Kestelman

2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

In this work, we evaluate aggressive undervolting, i.e., voltage scaling below the nominal level to reduce the energy consumption of Field Programmable Gate Arrays (FPGAs). Usually, voltage guardbands are added by chip vendors to ensure the worst-case process and environmental scenarios. Through experimenting on several FPGA architectures, we measure this voltage guardband to be on average 39% of the nominal level, which in turn, delivers more than an order of magnitude power savings. However, further undervolting below the voltage guardband may cause reliability issues as the result of the circuit delay increase, i.e., start to appear faults. We extensively characterize the behavior of these faults in terms of the rate, location, type, as well as sensitivity to environmental temperature, with a concentration of on-chip memories, or Block RAMs (BRAMs). Finally, we evaluate a typical FPGA-based Neural Network (NN) accelerator under low-voltage BRAM operations. In consequence, the substantial NN energy savings come with the cost of NN accuracy loss. To attain power savings without NN accuracy loss, we propose a novel technique that relies on the deterministic behavior of undervolting faults and can limit the accuracy loss to 0.1% without any timing-slack overhead.• This paper is the first effort to empirically study aggressive voltage underscaling of FPGAs below the standard nominal level. Through experimenting on four FPGA platforms, we confirm a large guardband, which is measured to be on average 39% of the nominal level for © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

show abstract

Section: B Recent Related Studies On Nnsmentioning

confidence: 99%

Comprehensive Evaluation of Supply Voltage Underscaling in FPGA on-Chip Memories

Salami

Ünsal

Kestelman

2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

show abstract

“…Cheng et al [25], Guo et al [49], Cheng et al [24] and Sze et al [136] surveyed algorithms for DNN compression and acceleration. Of these, Cheng et al [24] brie y evaluated system-level designs for FPGA implementation. Guo et al only surveyed quantisation methods; weight reduction was not mentioned.…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Network Approximation for Custom Hardware

et al. 2019

View full text Add to dashboard Cite

LondonDeep neural networks have proven to be particularly e ective in visual and audio recognition tasks. Existing models tend to be computationally expensive and memory intensive, however, and so methods for hardwareoriented approximation have become a hot topic. Research has shown that custom hardware-based neural network accelerators can surpass their general-purpose processor equivalents in terms of both throughput and energy e ciency. Application-tailored accelerators, when co-designed with approximation-based network training methods, transform large, dense and computationally expensive networks into small, sparse and hardware-e cient alternatives, increasing the feasibility of network deployment. In this article, we provide a comprehensive evaluation of approximation methods for high-performance network inference along with in-depth discussion of their e ectiveness for custom hardware implementation. We also include proposals for future research based on a thorough analysis of current trends. is article represents the rst survey providing detailed comparisons of custom hardware accelerators featuring approximation for both convolutional and recurrent neural networks, through which we hope to inspire exciting new developments in the eld.

show abstract

“…This paper uses the CNN algorithm; as one of the deep learning models, its multilevel structure of local perception and value sharing can accommodate high-dimensional data more accurately and quickly. Due to the characteristics of self-learning, especially the results are stable in the field of image and signal recognition, and there is no additional feature engineering requirement [33].…”

Section: Introductionmentioning

confidence: 99%

Recognition and Classification of Incipient Cable Failures Based on Variational Mode Decomposition and a Convolutional Neural Network

2019

View full text Add to dashboard Cite

To avoid power supply hazards caused by cable failures, this paper presents an approach of incipient cable failure recognition and classification based on variational mode decomposition (VMD) and a convolutional neural network (CNN). By using VMD, the original current signal is decomposed into seven modes with different center frequencies. Then, 42 features are extracted for the seven modes and used to construct a feature vector as input of the CNN to classify incipient cable failure through deep learning. Compared with using the original signals directly as the CNN input, the proposed approach is more efficient and robust. Experiments on different classifiers, namely, the decision tree (DT), K-nearest neighbor (KNN), BP neural network (BP) and support vector machine (SVM), and show that the CNN outperforms the other classifiers in terms of accuracy.

show abstract

Recent advances in efficient computation of deep convolutional neural networks

Cited by 210 publications

References 85 publications

Comprehensive Evaluation of Supply Voltage Underscaling in FPGA on-Chip Memories

Comprehensive Evaluation of Supply Voltage Underscaling in FPGA on-Chip Memories

Deep Neural Network Approximation for Custom Hardware

Recognition and Classification of Incipient Cable Failures Based on Variational Mode Decomposition and a Convolutional Neural Network

Contact Info

Product

Resources

About