Learning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss

Abstract

Reducing bit-widths of activations and weights of deep networks makes it efficient to compute and store them in memory, which is crucial in their deployments to resourcelimited devices, such as mobile phones. However, decreasing bit-widths with quantization generally yields drastically degraded accuracy. To tackle this problem, we propose to learn to quantize activations and weights via a trainable quantizer that transforms and discretizes them. Specifically, we parameterize the quantization intervals and obtain their optimal values by directly minimizing the task loss of the network. This quantization-interval-learning (QIL) allows the quantized networks to maintain the accuracy of the fullprecision (32-bit) networks with bit-width as low as 4-bit and minimize the accuracy degeneration with further bitwidth reduction (i.e., 3 and 2-bit). Moreover, our quantizer can be trained on a heterogeneous dataset, and thus can be used to quantize pretrained networks without access to their training data. We demonstrate the effectiveness of our trainable quantizer on ImageNet dataset with various network architectures such as ResNet-18, -34 and AlexNet, on which it outperforms existing methods to achieve the stateof-the-art accuracy.

Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks

“…But lower bit quantization faces more challenges on accuracy. To address this problem, [7,20] try to optimize the clipping value or quantization interval for the task specific loss in an end-to-end manner. [36,46,39] apply different techniques to find optimal bit-widths for each layer.…”

“…[36,46,39] apply different techniques to find optimal bit-widths for each layer. [42] and [20] optimize the training process with incremental and progressive quantization. [29] and [16] adjust the network structure to adapt to quantization.…”

“…To address * corresponding author this problem, the quantization technique has emerged as a promising network compression solution and achieved substantial progress in recent years. It can largely reduce the network storage and meanwhile accelerate the inference speed using different types of quantizers, mainly including binary/ternary [8,9,16,23,45,25,33], uniform [46,28,6,26,43,18,36,42,20,7,2,39,21] and nonuniform [44,35,13,5,41,30,38,37,3].…”

“…Both operations contribute to the quantization loss. Therefore, to alleviate the performance degradation, it is also important to find an appropriate clipping range and make a balance between clipping and rounding [7,20].…”

Domain Wall Memory-Based Design of Deep Neural Network Convolutional Layers

“…As shown in the figure, when the stride changes from 1 to 2, the number of redundant cells is fixed to 1, while the distance between two adjacent PEs (dPEs) is doubled. Although large stride or odd number of input feature bit-width can degrade the effectiveness of the proposed DWM-based cell array, since stride is generally less than 3 [21] and the 4-bit input and weight configurations can present the full-precision accuracy [28], [29] for recent CNNs, the proposed systolic DWM-based array is effectively reconfigurable for the different layer configuration.…”