EBPC: Extended Bit-Plane Compression for Deep Neural Network Inference and Training Accelerators

Abstract

In the wake of the success of convolutional neural networks in image classification, object recognition, speech recognition, etc., the demand for deploying these compute-intensive ML models on embedded and mobile systems with tight power and energy constraints at low cost, as well as for boosting throughput in data centers, is growing rapidly. This has sparked a surge of research into specialized hardware accelerators. Their performance is typically limited by I/O bandwidth, power consumption is dominated by I/O transfers to off-chip memory, and on-chip memories occupy a large part of the silicon area.We introduce and evaluate a novel, hardware-friendly, and lossless compression scheme for the feature maps present within convolutional neural networks. We present hardware architectures and synthesis results for the compressor and decompressor in 65 nm. With a throughput of one 8-bit word/cycle at 600 MHz, they fit into 2.8 kGE and 3.0 kGE of silicon area, respectivelytogether the size of less than seven 8-bit multiply-add units at the same throughput.We show that an average compression ratio of 5.1× for AlexNet, 4× for VGG-16, 2.4× for ResNet-34 and 2.2× for MobileNetV2 can be achieved-a gain of 45-70% over existing methods. Our approach also works effectively for various number formats, has a low frame-to-frame variance on the compression ratio, and achieves compression factors for gradient map compression during training that are even better than for inference.

Transform-Based Feature Map Compression for CNN Inference

“…Activations extracted by ReLU provide a known sparsity [10], thus several sparsity-based coding formats are designed to compress feature maps: Coordinate (COO) [25], Bitmap [26], Run-Length Coding [27] etc.. Several methods based on these coding formats describe their hardware architecture to reduce computational cost: Cnvlutin [11], SCNN [12], Eyeriss [13], EIE [14], etc.. Cambricon-SE [28] tries to use Huffman and LZW to improve the compression ratio of feature maps.…”

“…As a result, the large data of feature maps on CNN have to be repeatedly transferred between on-chip and off-chip memory, which greatly increases the power consumption and the data transfer bandwidth. Therefore, it is a topic worth exploring to reduce this power consumption and latency by compressing feature maps to improve the performance of CNN models on specialized hardware [10].…”

DNNZip: Selective Layers Compression Technique in Deep Neural Network Accelerators

“…The main drawback of compression representation learning based approaches is that they alter the DNN model and then require a retraining phase. EBPC [6] is a hardware-friendly and lossless compression scheme for the feature maps present within CNNs. However it is limited on the compression of the feature maps although the model parameters/weights are responsible for a major fraction of the overall memory/communication traffic (see Fig.…”

“…Current DNN models rely on millions or even billions of parameters, thus exacerbating the role played by the communication and memory sub-systems for moving such high data volume from the main memory into the accelerator and then to its many PEs. Thus, the performance and energy figures of a DNN accelerator are severely affected by the communication and memory sub-system [6], [7]. Fig.…”

ChewBaccaNN: A Flexible 223 TOPS/W BNN Accelerator

“…However, transmitting the data comes at a high energy cost, introduces privacy concerns, requires expensive infrastructure, and has high latency. Alternatively, the challenge of analyzing the data near the sensor can be tackled by a combination of algorithmic optimizations to allow working with reduced arithmetic precision, and hardware acceleration [3], [4] with various techniques to maximize energy efficiency, such as minimizing off-accelerator data transfers [5]- [7].…”