Robust quantization of deep neural networks

Kim, Young-Seok; Lee, Junyeol; Kim, Young-Hoon; Seo, Jiwon

doi:10.1145/3377555.3377900

Cited by 2 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of accumulators (at least three) tied to each multiply and accumulate (MAC) processing element (PE) might require more hardware resource for implementation, not to mention that existing CNN accelerators usually implements large number of PE in parallel to achieve high performance computation. In the work by Kim et al [22] and Cai et al [23], they proposed efficient search algorithm to find the optimal bitwidth for each of the layer weights. This scheme results in mixed precision for the network weights, where each layer might be represented using a different bitwidth.…”

Section: B Motivation: Why Another Level Of Quantization?mentioning

confidence: 99%

DoubleQExt: Hardware and Memory Efficient CNN Through Two Levels of Quantization

See

Ng²,

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

To fulfil the tight area and memory constraints in IoT applications, the design of efficient Convolutional Neural Network (CNN) hardware becomes crucial. Quantization of CNN is one of the promising approach that allows the compression of large CNN into a much smaller one, which is very suitable for IoT applications. Among various proposed quantization schemes, Power-of-two (PoT) quantization enables efficient hardware implementation and small memory consumption for CNN accelerators, but requires retraining of CNN to retain its accuracy. This paper proposes a two-level post-training static quantization technique (DoubleQ) that combines the 8-bit and PoT weight quantization. The CNN weight is first quantized to 8-bit (level one), then further quantized to PoT (level two). This allows multiplication to be carried out using shifters, by expressing the weights in their PoT exponent form. DoubleQ also reduces the memory storage requirement for CNN, as only the exponent of the weights is needed for storage. However, DoubleQ trades the accuracy of the network for reduced memory storage. To recover the accuracy, a selection process (DoubleQExt) was proposed to strategically select some of the less informative layers in the network to be quantized with PoT at the second level. On ResNet-20, the proposed DoubleQ can reduce the memory consumption by 37.50% with 7.28% accuracy degradation compared to 8-bit quantization. By applying DoubleQExt, the accuracy is only degraded by 1.19% compared to 8-bit version while achieving a memory reduction of 23.05%. This result is also 1% more accurate than the state-of-the-art work (SegLog). The proposed DoubleQExt also allows flexible configuration to trade off the memory consumption with better accuracy, which is not found in the other state-of-the-art works. With the proposed two-level weight quantization, one can achieve a more efficient hardware architecture for CNN with minimal impact to the accuracy, which is crucial for IoT applications.

show abstract

Section: B Motivation: Why Another Level Of Quantization?mentioning

confidence: 99%