Hybrid Binary Networks: Optimizing for Accuracy, Efficiency and Memory

Abstract

Binarization is an extreme network compression approach that provides large computational speedups along with energy and memory savings, albeit at significant accuracy costs. We investigate the question of where to binarize inputs at layer-level granularity and show that selectively binarizing the inputs to specific layers in the network could lead to significant improvements in accuracy while preserving most of the advantages of binarization. We analyze the binarization tradeoff using a metric that jointly models the input binarization-error and computational cost and introduce an efficient algorithm to select layers whose inputs are to be binarized. Practical guidelines based on insights obtained from applying the algorithm to a variety of models are discussed. Experiments on Imagenet dataset using AlexNet and ResNet-18 models show 3-4% improvements in accuracy over fully binarized networks with minimal impact on compression and computational speed. The improvements are even more substantial on sketch datasets like TU-Berlin, where we match state-of-the-art accuracy as well, getting over 8% increase in accuracies. We further show that our approach can be applied in tandem with other forms of compression that deal with individual layers or overall model compression (e.g., SqueezeNets). Unlike previous quantization approaches, we are able to binarize the weights in the last layers of a network, which often have a large number of parameters, resulting in significant improvement in accuracy over fully binarized models.

A Review of Binarized Neural Networks

“…Prabhu at al. [36] also explore the partial binarization. Instead of separating out individual kernels in a convolutional layer, each layer is analyzed as a whole.…”

Exploring the Connection Between Binary and Spiking Neural Networks

“…The obvious comparison point of this paper would be recent efforts at training quantized networks with bit-precision greater than single bit. There have been a multitude of approaches (Li et al, 2016 ; Zhou et al, 2016 , 2017 ; Choi et al, 2018 ; Deng et al, 2018 ; Zhang et al, 2018 ) with recent efforts aimed at designing networks with hybrid precision where the bit-precision of each layer of the network can vary (Prabhu et al, 2018 ; Wu et al, 2018 ; Chakraborty et al, 2019 ; Wang et al, 2019 ). However, in order to support variable bit-precision for each layer, the underlying hardware would need to be designed accordingly to handle mixed-precision (which usually is characterized by much higher area, latency and power consumption than BNN hardware accelerators.…”

GuCNet: A Guided Clustering-based Network for Improved Classification

“…Likewise, we achieve superior results (an improvement of over 6%) in comparison to contemporary techniques even for the TU-Berlin dataset when the baseline accuracy for the guide dataset is not as high as the MNIST dataset and the baseline accuracy for sketch data is only 69.90%. Table III also shows that Prabhu et al in [17] conducted experiments to quantify the performance of human subjects for the recognition of the image instances in the TU Berlin dataset. It was seen that even the humans could correctly classify the dataset with just 73.10% accuracy.…”

“…AlexNet-SVM [9] 67.10% AlexNet-Sketch [9] 68.60% Sketch-A-Net SC [15] 72.20% Sketch-A-Net-Hybrid [15] 73.10% ResNet18-Hybrid [16] 73.80% Humans [17] 73.10% Sketch-A-Net-Hybrid 2 77.00% Sketch-A-Net 2 77.00% Alexnet-FC-GRU [17] 79.95% Zhang et al [18] 82.95% Baseline 69.90% GuCNet (Prototype) 86.63% GuCNet (Texture) 89.26% H max by a factor of two, which reduces the separability among the prototypes. In the third case we further reduce the Hamming distance (H) to the minimum, i.e., H = 2 that corresponds to just one-hot coded prototype.…”