RNA: An Accurate Residual Network Accelerator for Quantized and Reconstructed Deep Neural Networks

Abstract

With the continuous refinement of Deep Neural Networks (DNNs), a series of deep and complex networks such as Residual Networks (ResNets) show impressive prediction accuracy in image classification tasks. Unfortunately, the structural complexity and computational cost of residual networks make hardware implementation difficult. In this paper, we present the quantized and reconstructed deep neural network (QR-DNN) technique, which first inserts batch normalization (BN) layers in the network during training, and later removes them to facilitate efficient hardware implementation. Moreover, an accurate and efficient residual network accelerator (RNA) is presented based on QR-DNN with batch-normalization-free structures and weights represented in a logarithmic number system. RNA employs a systolic array architecture to perform shift-and-accumulate operations instead of multiplication operations. QR-DNN is shown to achieve a 1% ∼ 2% improvement in accuracy over existing techniques, and RNA over previous best fixed-point accelerators. An FPGA implementation on a Xilinx Zynq XC7Z045 device achieves 804.03 GOPS, 104.15 FPS and 91.41% top-5 accuracy for the ResNet-50 benchmark, and state-of-the-art results are also reported for AlexNet and VGG. key words: deep learning, residual networks, software-hardware codesign, batch-normalization layers, FPGA

Towards efficient deep neural network inference and training for ubiquitous AI

“…The rapid development of DNNs in recent years has led them to become a core enabler for a broad spectrum of application areas, such as computer vision, natural language processing, medical engineering, autonomous driving, and virtual reality. Meanwhile, there is a rising trend showing that the deployment of these applications is shifting from traditional cloud computing platforms (e.g., servers and supercomputers) to edge devices (e.g., mobile and handheld platforms) whose power efficiency is one of the major constraints [148][149][150][151][152][153][154]. Field Programmable Gate Arrays (FPGAs) and mobile devices, as the two most popular substrates in edge devices 1 , have established their dominance through the delivery of promising energy/power efficiency and performance.…”

“…On FPGAs, weight quantization is a natural fit. Besides storage reduction, the additional benefits include (1) the DSP on FPGA can support multiple multiply-and-accumulate (MAC) computations with appropriate weight (and activation) quantization, and (2) the look-up table (LUT) computing resources can support low-precision computing [150,[178][179][180][181][182][183][184][185]. Low-bit-width fixedpoint quantization is achieved in [179] through greedy solution to determine the radix position of each layer for quantization, and in [182] with a hybrid quantization scheme that allows different bit-widths for weights to provide more flexibility.…”

“…Binarized Neural Networks (BNNs) can be implemented with XNOR gates to execute multiplications [178,180,181], and replacing zero padding with odd-even padding enables it to realize a fully binarized neural network accelerator [181]. The implementation with P2 quantization has been explored in [150].…”