Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation

Abstract

The nonuniform quantization strategy for compressing neural networks usually achieves better performance than its counterpart, i.e., uniform strategy, due to its superior representational capacity. However, many nonuniform quantization methods overlook the complicated projection process in implementing the nonuniformly quantized weights/activations, which incurs non-negligible time and space overhead in hardware deployment. In this study, we propose Nonuniform-to-Uniform Quantization (N2UQ), a method that can maintain the strong representation ability of nonuniform methods while being hardware-friendly and efficient as the uniform quantization for model inference. We achieve this through learning the flexible inequidistant input thresholds to better fit the underlying distribution while quantizing these real-valued inputs into equidistant output levels. To train the quantized network with learnable input thresholds, we introduce a generalized straight-through estimator (G-STE) for intractable backward derivative calculation w.r.t. threshold parameters. Additionally, we consider entropy preserving regularization to further reduce information loss in weight quantization. Even under this adverse constraint of imposing uniformly quantized weights and activations, our N2UQ outperforms state-of-the-art nonuniform quantization methods by 0.7∼1.8% on ImageNet, demonstrating the contribution of N2UQ design. Code will be made publicly available.

Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference

“…These methods focus on quantizing most, if not all network layers to the same uniform bit-width. While this has been shown to be effective for recovering full precision accuracy for higher bit widths, using extremely low precision still leads to significant accuracy degradation (Courbariaux et al, 2015;Esser et al, 2015;Rastegari et al, 2016;Zhou et al, 2016;McKinstry et al, 2019;Esser et al, 2020;Liu et al, 2021c). To further push the envelope of maximizing throughput and minimizing memory footprint while maintaining task performance, mixed precision quantization methods have emerged with the goal of optimizing the bit-width of each layer independently to maximize overall network performance (Dong et al, 2019;Yao et al, 2021;Chen et al, 2021).…”