Fixed point optimization of deep convolutional neural networks for object recognition

Abstract

Deep convolutional neural networks have shown promising results in image and speech recognition applications. The learning capability of the network improves with increasing depth and size of each layer. However this capability comes at the cost of increased computational complexity. Thus reduction in hardware complexity and faster classification are highly desired. This work proposes an optimization method for fixed point deep convolutional neural networks. The parameters of a pre-trained high precision network are first directly quantized using L2 error minimization. We quantize each layer one by one, while other layers keep computation with high precision, to know the layer-wise sensitivity on word-length reduction. Then the network is retrained with quantized weights. Two examples on object recognition, MNIST and CIFAR-10, are presented. Our results indicate that quantization induces sparsity in the network which reduces the effective number of network parameters and improves generalization. This work reduces the required memory storage by a factor of 1/10 and achieves better classification results than the high precision networks.Index Terms-convolutional neural network, quantization, word length optimization, sparsity

Proteus

“…Seide et al showed how to quantize the gradients using only one bit per value to train a DNN using Stochastic Gradient Descent without loss of accuracy [35]. Anwar et al also consider using di↵erent precisions per-layer for LeNet and Convnet [2]. They use linear quantization and retraining to achieve smaller precisions and improve the accuracy of the network.…”

“…Recently, Judd et al and Anwar et al observed that the required representation length varies significantly across DNN layers with the worst-case representation length being much longer than the average length needed [21,2]. This behavior persists across multiple data inputs suggesting that a perlayer choice can be made safely without sacrificing overall accuracy.…”

Retraining-free methods for fast on-the-fly pruning of convolutional neural networks

“…Thus, it is to no surprise that several techniques exist for "pruning" or "sparsifying" CNNs (i.e., forcing some model weights to 0) to both compress the model and to save computations during inference. Examples of these techniques include: iterative pruning and retraining ( [3,9,4,27,20]), Huffman coding ( [6]), exploiting granularity ( [18,5]), structural pruning of network connections ( [32,19,1,22]), and Knowledge Distillation (KD) ( [11]).…”

“…In this paper, we seek to answer some fundamental questions that relate to the trade-off between sparsity and inference accuracy: (1) To what extent a CNN can be sparsified without retraining while maintaining a reasonable inference accuracy, (2) What are good model-independent methods for sparsifying CNNs, and (3) Can such sparsification benefit from autotuning [8,2]. We focus on sparsification leaving the actual exploitation of the resulting sparsity to future work.…”

Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?

“…Retrain each BNN for maximally M epochs. 2 Ensemble Pass: 3 for k=1 to K do 4 Sampling a new training set given weight u i of each example i; 5 for epoch=1 to M do 6 Forward Pass: 7 for l=1 to L do 8 for each filter in l-th layer do…”