A Dual-Mode 2-bit Uniform Scalar Quantizer for Data with Laplacian Distribution

Perić, Zoran; Denic, Bojan; Jovanović, Aleksandra; Savic, Milan; Vučić, Nikola; Nikolic, Anastasija

doi:10.1109/telfor52709.2021.9653253

Cited by 1 publication

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Actually, we determine the key parameter Δ(σ ref ) =σ ref •Δ = Δ such that minimal mean-squared error (MSE) distortion or equivalently maximal signal to quantization noise ratio (SQNR) is achieved. MSE distortion of the 2-bit USQ is given by [21]:…”

Section: Figurementioning

confidence: 99%

“…Let us now consider the performance of the designed 2-bit USQ under variance-mismatch condition, that is, under condition that 2-bit USQ quantizes the Laplacian data having variance different from the designed one (σ 2 ≠ σ ref 2 = 1). Using PDF defined with (2), for distortion is obtained [21,26]:…”

Section: A 2-bit Usq In Variance-mismatch Scenariomentioning

confidence: 99%

“…Hence, it is of special importance to develop an efficient quantization scheme, because it can significantly contribute to the performance of quantized NN. Most of the available quantization schemes are based on fixed-length coding, where different codewords length are employed including > 8 bits [23], 8 bits [1], 4 bits [3], 2 bits [4,5,20,21,26] or even 1 bit [8,22,27,28]. It was reported that quantized NN provides a negligible decreasing of performance with respect to full precision NN when high bit lengths (≥ 8 bits) have been employed, where compression ratio ≤ 4 times was achieved.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, we investigate the variance-mismatch effect in case of 2-bit USQ and propose an adequate measure to deal with it. Note also that the practical significance of 2-bit quantizers is large, as they have already been successively applied in several data processing applications such as speech coding [18,19], image coding [11,29] or, as mentioned, NN compression [4,5,20,21,26]. Regarding the implementation in NN compression, it was shown in [4,5,26] that 2-bit USQ can enable a good trade-off between performance and size of NN.…”

Section: Introductionmentioning

confidence: 99%

“…Instead of searching the content of a block as in [24,25], in this papers selection among the two USQs is done based on the estimated block variance. The idea originates from [21], but few progressive steps are performed in this paper. We change a heuristically de-termined step size value (key parameter) of the initial 2-bit USQ by the one which is obtained by maximization of performance for the particular variance, further we determine the best value for scaling factor (used to scale the initial step size) and also we propose an iterative rule for determining the best switching threshold of the dual-mode USQ.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Performance Analysis of a 2-bit Dual-Mode Uniform Scalar Quantizer for Laplacian Source

Perić

Denic

Jovanović

et al. 2022

ITC

View full text Add to dashboard Cite

The main issue when dealing with the non-adaptive scalar quantizers is their sensitivity to variance-mismatch, the effect that occurs when the data variance differs from the one used for the quantizer design. In this paper, we consider the influence of that effect in low-rate (2-bit) uniform scalar quantization (USQ) of Laplacian source and also we propose adequate measure to suppress it. Particularly, the approach we propose represents the upgraded version of the previous approaches used to improve performance of the single quantizer. It is based on dual-mode quantization that combines two 2-bit USQs (with adequately chosen parameters) to process input data, selected by applying the special rule. Analysis conducted in theoretical domain has shown that the proposed approach is less sensitive to variance-mismatch, making the dual-mode USQ more efficient in terms of robustness than the single USQ. Also, a gain is achieved compared to other 2-bit quantizer solutions. Experimental results are also provided for quantization of weights of the multi-layer perceptron (MLP) neural network, where good matching with the theoretical results is observed. Due to these achievements, we believe that the solution we propose can be a good choice for compression of non-stationary data modeled by Laplacian distribution, such as neural network parameters.

show abstract

Section: Figurementioning

confidence: 99%

Section: A 2-bit Usq In Variance-mismatch Scenariomentioning

confidence: 99%