Efficient hardware acceleration of CNNs using logarithmic data representation with arbitrary log-base

Vogel, Sebastian; Liang, Mengyu; Guntoro, Andre; Stechele, Walter; Ascheid, Gerd

doi:10.1145/3240765.3240803

Cited by 50 publications

(37 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some authors [20], [21] proposed logarithmic-scaled quantization for efficient encoding of edge weight parameters to reduce the memory size requirements of an ANN. In this paper, we apply logarithmic-scaled quantization to neuron activation, the output value of each neuron, instead of edge weights, in order to achieve efficient ANN-to-SNN conversion.…”

Section: Ann-to-snn Conversionmentioning

confidence: 99%

Efficient Spiking Neural Networks With Logarithmic Temporal Coding

Zhang

Zheng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

A Spiking Neural Network (SNN) can be trained indirectly by first training an Artificial Neural Network (ANN) with the conventional backpropagation algorithm, then converting it into an equivalent SNN. To reduce the computational cost of the resulting SNN as measured by the number of spikes, we present Logarithmic Temporal Coding (LTC), where the number of spikes used to encode an activation grows logarithmically with the activation value; and the accompanying Exponentiate-and-Fire (EF) neuron model, which only involves efficient bit-shift and addition operations. Moreover, we improve the training process of ANN to compensate for approximation errors due to LTC. Experimental results indicate that the resulting SNN achieves competitive performance in terms of classification accuracy at significantly lower computational cost than related work. INDEX TERMS Spiking neural networks, temporal coding, neuromorphic computing.

show abstract

Section: Ann-to-snn Conversionmentioning

confidence: 99%

Efficient Spiking Neural Networks With Logarithmic Temporal Coding

Zhang

Zheng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The technique proposed in [13] utilizes different bit-widths for each layer to reduce the errors in final output accuracy. To replace the computationally costly multiply-operation with bit-shifts, the works in [12], [14]- [18] have used the power of 2 quantizations for pretrained networks' parameters. However, most of these techniques require a fine-tuning (retraining) step to reduce the errors induced due to quantization.…”

Section: Related Workmentioning

confidence: 99%

ALigN: A Highly Accurate Adaptive Layerwise Log_2_Lead Quantization of Pre-Trained Neural Networks

et al. 2020

View full text Add to dashboard Cite

Deep Neural Networks are one of the machine learning techniques which are increasingly used in a variety of applications. However, the significantly high memory and computation demands of deep neural networks often limit their deployment on embedded systems. Many recent works have considered this problem by proposing different types of data quantization schemes. However, most of these techniques either require post-quantization retraining of deep neural networks or bear a significant loss in output accuracy. In this paper, we propose a novel and scalable technique with two different modes for the quantization of the parameters of pre-trained neural networks. In the first mode, referred to as log_2_lead, we use a single template for the quantization of all parameters. In the second mode, denoted as ALigN, we analyze the trained parameters of each layer and adaptively adjust the quantization template to achieve even higher accuracy. Our technique significantly maintains the accuracy of the parameters and does not require retraining of the networks. Moreover, it supports quantization to an arbitrary bit-size. For example, compared to the single-precision floating-point numbers-based implementation, our proposed 8bit quantization technique generates only ∼ 0.2% and ∼ 0.1%, loss in the Top-1 and Top-5 accuracies respectively for VGG-16 network using ImageNet dataset. We have observed similar minimal losses in the Top-1 and Top-5 accuracies for AlexNet and Resnet-18 using the proposed quantization scheme for the 8-bit range. Our proposed quantization technique also provides a higher mean intersection over union for semantic segmentation when compared with state-of-the-art quantization techniques. The proposed technique represents parameters in powers of 2, thereby eliminating the need for resource-computationally intensive multiplier units for the hardware accelerators of the neural networks. We also present a design for implementing the multiplication operation using bit-shifts and addition for the proposed quantization technique.

show abstract

“…It also gives accuracy comparison between linear and log quantization. [12] proposes an accelerator design using arbitrary log base. It, however, does not utilize the low hardware overhead of the log-based PE and instead rely on linear PE arrangements.…”

Section: Related Workmentioning

confidence: 99%

“…Basic log-based multiplication operation is performed in a single thread. Assuming we have two log quantized values, w q ' and a q ', representing the original weight (w q ) and the activation input (a q ) respectively, the multiplication of these values in log domain can be carried out as: [12] showed a method to implement the exponential in equation (5) in hardware by decomposing the exponent into its integer and fractional part as:…”

Section: Hardware Architecture 41 Top-levelmentioning

confidence: 99%