ALWANN: Automatic Layer-Wise Approximation of Deep Neural Network Accelerators without Retraining

Abstract

The state-of-the-art approaches employ approximate computing to reduce the energy consumption of DNN hardware. Approximate DNNs then require extensive retraining afterwards to recover from the accuracy loss caused by the use of approximate operations. However, retraining of complex DNNs does not scale well. In this paper, we demonstrate that efficient approximations can be introduced into the computational path of DNN accelerators while retraining can completely be avoided. ALWANN provides highly optimized implementations of DNNs for custom low-power accelerators in which the number of computing units is lower than the number of DNN layers. First, a fully trained DNN (e.g., in TensorFlow) is converted to operate with 8-bit weights and 8-bit multipliers in convolutional layers. A suitable approximate multiplier is then selected for each computing element from a library of approximate multipliers in such a way that (i) one approximate multiplier serves several layers, and (ii) the overall classification error and energy consumption are minimized. The optimizations including the multiplier selection problem are solved by means of a multiobjective optimization NSGA-II algorithm. In order to completely avoid the computationally expensive retraining of DNN, which is usually employed to improve the classification accuracy, we propose a simple weight updating scheme that compensates the inaccuracy introduced by employing approximate multipliers. The proposed approach is evaluated for two architectures of DNN accelerators with approximate multipliers from the open-source "EvoApprox" library, while executing three versions of ResNet on CIFAR-10. We report that the proposed approach saves 30% of energy needed for multiplication in convolutional layers of ResNet-50 while the accuracy is degraded by only 0.6% (0.9% for the ResNet-14). The proposed technique and approximate layers are available as an open-source extension of TensorFlow at https://github.com/ehw-fit/tf-approximate.Index Terms-approximate computing, deep neural networks, computational path, ResNet, CIFAR-10

Design Automation of Approximate Circuits With Runtime Reconfigurable Accuracy

“…We select Neural Network inference as our testcase and we use the ResNet-8 network [59] (7 convolution layers) trained with the CIFAR-10 image classification dataset. The NN is quantized at 8 bits to avoid floating point operations [60]. First, we use RETSINA to generate two approximate reconfigurable 8-bit multipliers.…”

“…The accuracy levels of RM1 and RM2 are selected targeting high inference accuracy. To achieve this, we used [31] and [60] to profile how the multiplier's error impacts ResNet's accuracy. Next, we extend [60] and integrate our reconfigurable multipliers RM1 and RM2.…”

“…To achieve this, we used [31] and [60] to profile how the multiplier's error impacts ResNet's accuracy. Next, we extend [60] and integrate our reconfigurable multipliers RM1 and RM2. In [60], approximate multipliers are used to perform the multiplications of each convolution layer.…”

“…Next, we extend [60] and integrate our reconfigurable multipliers RM1 and RM2. In [60], approximate multipliers are used to perform the multiplications of each convolution layer. However, since different layers have different accuracy requirement, [60] employs an heterogeneous architecture that comprises several fixed approximate multiplier types.…”

Low-power implementation of Mitchell's approximate logarithmic multiplication for convolutional neural networks

“…The works presented in [10], [13], [20], [36], [37] had used logic minimization to create the optimal approximate multipliers for each network model. Logic minimization intentionally flips bits in the logic to reduce the size of the operators, and these techniques use heuristics to find the optimal targets.…”