LSQ+: Improving low-bit quantization through learnable offsets and better initialization

Bhalgat, Yash; Lee, Jinwon; Nagel, Markus; Blankevoort, Tijmen; Kwak, Nojun

doi:10.1109/cvprw50498.2020.00356

Cited by 119 publications

(115 citation statements)

References 9 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…In recent years researchers have been able to push more and more the limits of quantization. In particular, there is a growing corpora of work showing that extreme quantization of the weights down to 4bit is possible, while minimally affecting the prediction accuracy of the network [8]- [14]. 4bit quantization offers directly 8× compression gains and similar improvements in computational efficiency.…”

Section: A Techniques For Reducing the Information Content Of The Dnns Parametersmentioning

confidence: 99%

FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4Bit-Compact Multilayer Perceptrons

Wiedemann

Shivapakash

Becking

et al. 2021

IEEE Open J. Circuits Syst.

View full text Add to dashboard Cite

With the growing demand for deploying Deep Learning models to the "edge", it is paramount to develop techniques that allow to execute state-of-the-art models within very tight and limited resource constraints. In this work we propose a software-hardware optimization paradigm for obtaining a highly efficient execution engine of deep neural networks (DNNs) that are based on fully-connected layers. The work's approach is centred around compression as a means for reducing the area as well as power requirements of, concretely, multilayer perceptrons (MLPs) with high predictive performances. Firstly, we design a novel hardware architecture named FantastIC4, which (1) supports the efficient on-chip execution of multiple compact representations of fully-connected layers and (2) minimizes the required number of multipliers for inference down to only 4 (thus the name). Moreover, in order to make the models amenable for efficient execution on FantastIC4, we introduce a novel entropyconstrained training method that renders them to be robust to 4bit quantization and highly compressible in size simultaneously. The experimental results show that we can achieve throughputs of 2.45 TOPS with a total power consumption of 3.6W on a Virtual Ultrascale FPGA XCVU440 device implementation, and achieve a total power efficiency of 20.17 TOPS/W on a 22nm process ASIC version. When compared to other state-of-the-art accelerators designed for the Google Speech Command (GSC) dataset, FantastIC4 is better by 51× in terms of throughput and 145× in terms of area efficiency (GOPS/mm 2 ).

show abstract

Section: A Techniques For Reducing the Information Content Of The Dnns Parametersmentioning

confidence: 99%

FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4Bit-Compact Multilayer Perceptrons

Wiedemann

Shivapakash

Becking

et al. 2021

IEEE Open J. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…Although non-uniform quantization provides a better performance for a wide range of input signal variances [6,7] and advanced dual-mode asymptotic solutions 2 of 17 are developed [8,9], simple uniform quantization [6,7,[10][11][12] is the first choice when the simplicity of the system is one of the major goals. Thus, uniform quantization has been widely applied for quantizing parameters of neural networks (i.e., for neural network compression) [13][14][15][16][17][18], and different solutions have been considered, e.g., using 8-bits [13], 4-bits [14], or 2-bits [15][16][17][18]; further, non-uniform quantization has also been used [19][20][21]. It has been found that quantizing network parameters using 8-bits [13] or 16-bits [19] enable slightly lower performance when compared to the full precision case, mainly due to the ability of quantizers to achieve high quality reconstructed data.…”

Section: Introductionmentioning

confidence: 99%

“…It has been found that quantizing network parameters using 8-bits [13] or 16-bits [19] enable slightly lower performance when compared to the full precision case, mainly due to the ability of quantizers to achieve high quality reconstructed data. Further, in the case of applying quantizers with smaller resolution, e.g., with 4-bits [14] or 2-bits [15][16][17][18]20,21], performance degradation has been observed; however, the achieved results are still comparable, accompanied with a significantly high level of compression. Eventually, significant attention was paid to the development of binary quantizer models to compress neural networks [22][23][24][25][26], whose attractiveness lies in the amount of compression that can be achieved, with a goal to preserve competitive performance achievements.…”

Section: Introductionmentioning

confidence: 99%

“…Note that the determination of the clipping factor for various quantizer solutions has been the subject of many research papers [14][15][16]26,29], implying the significance of this parameter. In addition to other research papers, e.g., [13][14][15][16][17][18][19][20][21]29], we perform several other analyses from the aspect of signal processing, including an analysis in the wide range of input signal variances and adaptation of the quantization model.…”

Section: Introductionmentioning

confidence: 99%

“…We analyze the effectiveness of the proposed adaptive 2-bit quantizer in a real environment by implementing it in a neural network compression task, and the obtained performance is compared with the performance of the full-precision network, as well as with the performance of other contemporary 2-bit quantization models, either uniform [17,18] or non-uniform ones [20,21]. The first neural network model adopted in this paper is multi-layer perceptron (MLP) [30], which represents a kind of simple feedforward artificial neural network.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Design of a 2-Bit Neural Network Quantizer for Laplacian Source

Perić

Savic²,

Simic

et al. 2021

Entropy

View full text Add to dashboard Cite

Achieving real-time inference is one of the major issues in contemporary neural network applications, as complex algorithms are frequently being deployed to mobile devices that have constrained storage and computing power. Moving from a full-precision neural network model to a lower representation by applying quantization techniques is a popular approach to facilitate this issue. Here, we analyze in detail and design a 2-bit uniform quantization model for Laplacian source due to its significance in terms of implementation simplicity, which further leads to a shorter processing time and faster inference. The results show that it is possible to achieve high classification accuracy (more than 96% in the case of MLP and more than 98% in the case of CNN) by implementing the proposed model, which is competitive to the performance of the other quantization solutions with almost optimal precision.

show abstract

ECQ$$^{\text {x}}$$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs

Becking

Dreyer

Samek

et al. 2022

xxAI - Beyond Explainable AI

View full text Add to dashboard Cite

The remarkable success of deep neural networks (DNNs) in various applications is accompanied by a significant increase in network parameters and arithmetic operations. Such increases in memory and computational demands make deep learning prohibitive for resource-constrained hardware platforms such as mobile devices. Recent efforts aim to reduce these overheads, while preserving model performance as much as possible, and include parameter reduction techniques, parameter quantization, and lossless compression techniques.In this chapter, we develop and describe a novel quantization paradigm for DNNs: Our method leverages concepts of explainable AI (XAI) and concepts of information theory: Instead of assigning weight values based on their distances to the quantization clusters, the assignment function additionally considers weight relevances obtained from Layer-wise Relevance Propagation (LRP) and the information content of the clusters (entropy optimization). The ultimate goal is to preserve the most relevant weights in quantization clusters of highest information content.Experimental results show that this novel Entropy-Constrained and XAI-adjusted Quantization (ECQ$$^{\text {x}}$$ x ) method generates ultra low-precision (2–5 bit) and simultaneously sparse neural networks while maintaining or even improving model performance. Due to reduced parameter precision and high number of zero-elements, the rendered networks are highly compressible in terms of file size, up to 103$$\times $$ × compared to the full-precision unquantized DNN model. Our approach was evaluated on different types of models and datasets (including Google Speech Commands, CIFAR-10 and Pascal VOC) and compared with previous work.

show abstract

LSQ+: Improving low-bit quantization through learnable offsets and better initialization

Cited by 119 publications

References 9 publications

FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4Bit-Compact Multilayer Perceptrons

FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4Bit-Compact Multilayer Perceptrons

Design of a 2-Bit Neural Network Quantizer for Laplacian Source

ECQ$$^{\text {x}}$$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs

Contact Info

Product

Resources

About