Neuron-by-Neuron Quantization for Efficient Low-Bit QNN Training

Abstract

Quantized neural networks (QNNs) are widely used to achieve computationally efficient solutions to recognition problems. Overall, eight-bit QNNs have almost the same accuracy as full-precision networks, but working several times faster. However, the networks with lower quantization levels demonstrate inferior accuracy in comparison to their classical analogs. To solve this issue, a number of quantization-aware training (QAT) approaches were proposed. In this paper, we study QAT approaches for two- to eight-bit linear quantization schemes and propose a new combined QAT approach: neuron-by-neuron quantization with straight-through estimator (STE) gradient forwarding. It is suitable for quantizations with two- to eight-bit widths and eliminates significant accuracy drops during training, which results in better accuracy of the final QNN. We experimentally evaluate our approach on CIFAR-10 and ImageNet classification and show that it is comparable to other approaches for four to eight bits and outperforms some of them for two to three bits while being easier to implement. For example, the proposed approach to three-bit quantization of the CIFAR-10 dataset results in 73.2% accuracy, while baseline direct and layer-by-layer result in 71.4% and 67.2% accuracy, respectively. The results for two-bit quantization for ResNet18 on the ImageNet dataset are 63.69% for our approach and 61.55% for the direct baseline.

4.6-Bit Quantization for Fast and Accurate Neural Network Inference on CPUs

“…The top 1 and top 5 accuracies for ImageNet are shown in Table 7. The best quantization parameters (N x , N w ) for 4.6-bit models were (23,23). As for CIFAR-10, eight-bit models have comparable accuracy to full precision ones, while four bits give considerably lower results.…”

“…Notably, recent research [22] has demonstrated the ability to train four-bit networks for some tasks with minimal accuracy loss. Additionally, there is an ongoing development of quantization-aware training (QAT) techniques for scenarios where training data are available [23,24]. These approaches show significant potential for advancing the field of computationally efficient recognition and facilitating the implementation of QNNs in practical applications.…”

“…There are 21 pairs (N x , N w ) which satisfy ( 6): (255, 3), (127, 5), (85, 7), (63, 9), (51, 11), (43,13), (37,15), (31,17), (29,19), (25,21), (23,23) and symmetrical ones. If we compute the average bitwidth required to store x and w as (log 2 N x + log 2 N w )/2, we obtain values in the range 4.51-4.79.…”

“…During fine-tuning, we used stochastic gradient descent with momentum 0.9 and learning rate 10 −5 . That procedure is similar to QAT algorithm [23].…”

Software System for Automatic Grading of Paper Tests

“…In the initial convolutional layers, L2 regularization is employed to penalize the error function more, aiming to reduce the complexity of the model and prevent overfitting. Along with data augmentation, all of the aforementioned measures are intended to reduce the model's variance [24,25].…”