Reducing bit-widths of activations and weights of deep networks makes it efficient to compute and store them in memory, which is crucial in their deployments to resourcelimited devices, such as mobile phones. However, decreasing bit-widths with quantization generally yields drastically degraded accuracy. To tackle this problem, we propose to learn to quantize activations and weights via a trainable quantizer that transforms and discretizes them. Specifically, we parameterize the quantization intervals and obtain their optimal values by directly minimizing the task loss of the network. This quantization-interval-learning (QIL) allows the quantized networks to maintain the accuracy of the fullprecision (32-bit) networks with bit-width as low as 4-bit and minimize the accuracy degeneration with further bitwidth reduction (i.e., 3 and 2-bit). Moreover, our quantizer can be trained on a heterogeneous dataset, and thus can be used to quantize pretrained networks without access to their training data. We demonstrate the effectiveness of our trainable quantizer on ImageNet dataset with various network architectures such as ResNet-18, -34 and AlexNet, on which it outperforms existing methods to achieve the stateof-the-art accuracy.

No abstract

The current paper is concerned with an effective method to quantize a spectrum envelope of a speech signal without having an inter-frame prediction. In this paper, we proposed a method referred to as dynamic bit allocation-split vector quantization (DBA-SVQ). The main feature of this structure is that it makes use of the ordering property of line spectral frequencies (LSF) and exploits multiple codebooks, normalization and the DBA technique. As a result, we can limit the dynamic range of LSF sub-vectors and allocate different numbers of bits in accordance with the range sizes to maximize the overall efficiency of quantization. The performance is compared with delta line spectral pairs (LSP) VQ, which is used in EVRC-B, demonstrating reduction in spectral distortion (SD).

No abstract

No abstract