Understanding the impact of precision quantization on the accuracy and energy of neural networks

Abstract

Recently, the posit numerical format has shown promise for DNN data representation and compute with ultralow precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and Fashion MNIST corpuses, where 16-bit posits outperform 16-bit floating point for end-to-end DNN training.Index Terms-Deep neural networks, low-precision arithmetic, posit numerical format I. INTRODUCTIONThe edge computing, offers a decentralized solution to cloud-based datacenters [1] and intelligence-at-the-edge of mobile networks. However, training on the edge is a challenge for many deep neural networks (DNNs). This arises due to the significant cost of multiply-and-accumulate (MAC) units, an ubiquitous operation in all DNNs. In a 45 nm CMOS process, energy consumption doubles from 16-bit floats to 32-bit floats for addition and by ∼4× for multiplication [2]. Memory access cost increases by ∼10× from 8 kB to 1 MB memory with a 64-bit cache [2]. In general, there is a gap between memory storage, bandwidth, compute requirements, and energy consumption of modern DNNs and hardware resources available on edge devices [3].An apparent solution to address this gap is to compress such networks, thus reducing the compute requirements to match putative edge resources. Several groups have proposed compressed new compute-and memory-efficient DNN architectures [4]-[6] and parameter-efficient neural networks, using methods such as DNN pruning [7], distillation [8], and low-precision arithmetic [9], [10]. Among these approaches, low-precision arithmetic is noted for its ability to reduce memory capacity, bandwidth, latency, and energy consumption associated with MAC units in DNNs and an increase in the level of data parallelism [9], [11], [12].The ultimate goal of low-precision DNN design is to reduce the original hardware complexity of the high-precision DNN model to a level suitable for edge devices without significantly degrading performance.To address the gaps in previous studies, we are motivated to study low-precision posit for DNN training on the edge. II. POSIT NUMERICAL FORMATAn alternative to IEEE-754 floating point numbers, posits were recently introduced and exhibit a tapered-precision char-

Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks

“…8-bit) to high-precision floating point (e.g. 32-bit) [14]. However, these works compare numerical formats with disparate bit-widths and thereby do not fairly provide a comprehensive, holistic study of the network efficiency.…”

Deep Positron: A Deep Neural Network Using the Posit Number System

“…After decoding inputs, multiplication and converting to fixed-point is performed similarly to that of floating point. Products are accumulated in a register, or quire in the posit literature, of width qsize as given by (4). qsize = 2 es+2 × (n − 2) + 2 + log 2 (k) , n ≥ 3 (4)…”

“…8-bit) to a floating point high-precision (e.g. 32-bit) [4]. The utility of these studies is limited -the comparisons are across numerical formats with different bit widths and do not provide a fair understanding of the overall system efficiency.…”

Optimizing Convolutional Neural Networks for Embedded Systems by Means of Neuroevolution

“…The most popular approach is to introduce approximate computing techniques to CNNs and benefit from the fact that the applications utilizing CNNs are highly error resilient (i.e., a huge reduction in energy consumption can be obtained for an acceptable loss in accuracy) [7]. Approximate implementations of CNNs are based on various techniques such as innovative hardware architectures of CNN accelerators, simplified data representation, pruning of less significant neurons, approximate arithmetic operations, approximate memory access, weight compression and "in memory" computing [11,7,2]. For example, employing the FX operations has many advantages such as reduced (i) power consumption per arithmetic operation, (ii) memory capacity needed to store the weights and (iii) processor-memory data transfer time.…”

“…Our objective is to design and optimize not only with respect to the classification error, but also with respect to hardware resources needed when the final (trained) CNN is implemented in an embedded system with limited resources. As energy-efficient machine learning is a highly desired technology, various approximate implementations of CNNs have been introduced [7,2]. Contrasted to the existing neuroevolutionary approaches trying to minimize the classification error as much as possible and assuming that CNN is executed using floating point (FP) operations on a Graphical Processing Unit (GPU) [3, arXiv:1910.06854v1 [cs.NE] 15 Oct 2019 1], our target is a highly optimized CNN whose major parts are executed with reduced precision in fixed point (FX) arithmetic operations.…”