Data-Free Quantization Through Weight Equalization and Bias Correction

Abstract

We introduce a data-free quantization method for deep neural networks that does not require fine-tuning or hyperparameter selection. It achieves near-original model performance on common computer vision architectures and tasks. 8-bit fixed-point quantization is essential for efficient inference in modern deep learning hardware architectures. However, quantizing models to run in 8-bit is a non-trivial task, frequently leading to either significant performance reduction or engineering time spent on training a network to be amenable to quantization. Our approach relies on equalizing the weight ranges in the network by making use of a scale-equivariance property of activation functions. In addition the method corrects biases in the error that are introduced during quantization. This improves quantization accuracy performance, and can be applied ubiquitously to almost any model with a straight-forward API call. For common architectures, such as the MobileNet family, we achieve state-of-the-art quantized model performance. We further show that the method also extends to other computer vision architectures and tasks such as semantic segmentation and object detection.

Towards Mitigating Device Heterogeneity in Federated Learning via Adaptive Model Quantization

“…In real-world end-device settings, dierent bit-widths are supported by dierent devices [19,20,23,27,30]. This hardware exibility allows for the ability of conguring the model with dierent quantization levels to match the variety of hardware congurations of clients' mobile or IoT devices.…”

AI Benchmark: All About Deep Learning on Smartphones in 2019

“…Disadvantages: Reducing the bit-width of the network weights (from 16 to 8 bits) leads to accuracy loss: in some cases, the converted model might show only a small performance degradation, while for some other tasks the resulting accuracy will be close to zero. Although a number of research papers dealing with network quantization were presented by Qualcomm [49,54] and Google [34,37], all showing decent accuracy results for many image classification models, there is no general recipe for quantizing arbitrary deep learning architectures. Thus, quantization is still more of a research topic, without working solutions for many AIrelated tasks (e.g., image-to-image mapping or various NLP problems).…”

Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence

“…OCS [20] instead splits a channel into two channels in which the weights and outputs are halved, which reduces the dynamic range of the outliers. DFQ [21] quantizes weights and activations to 8 bits by assuming that the inputs to the activations have a Gaussian distribution, so that a model can be used to equalize the dynamic range of the data being quantized, along with a correction to the bias introduced by quantization. Clipping-based approaches are used in [10], [22], [23], in which activations or weights are clipped prior to quantization.…”