ProxyBNN: Learning Binarized Neural Networks via Proxy Matrices

He, Xiaoliang; Mo, Zitao; Cheng, Ke; Xu, Weixiang; Hu, Qinghao; Wang, Peisong; Liu, Qingshan; Cheng, Jian

doi:10.1007/978-3-030-58580-8_14

Cited by 17 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By looking at the reported table, we can observe that our method outperforms all the previous quantization methods in top-1 accuracy. Specifically, we can achieve significant performance gain over the recent state-of-the-art methods 56.4 XNOR-Net++ [3] 57.1 IR-Net [41] 58.1 ProxyBNN [20] 58.7 RBNN [33] 59.9 BinaryDuo [28] 60. LSQ, QKD, and SAT on all comparing architectures.…”

Section: Imagenet Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer

2021

View full text Add to dashboard Cite

Quantizing weights and activations of deep neural networks is essential for deploying them in resourceconstrained devices, or cloud platforms for at-scale services. While binarization is a special case of quantization, this extreme case often leads to several training difficulties, and necessitates specialized models and training methods. As a result, recent quantization methods do not provide binarization, thus losing the most resource-efficient option, and quantized and binarized networks have been distinct research areas. We examine binarization difficulties in a quantization framework and find that all we need to enable the binary training are a symmetric quantizer, good initialization, and careful hyperparameter selection. These techniques also lead to substantial improvements in multibit quantization. We demonstrate our unified quantization framework, denoted as UniQ, on the ImageNet dataset with various architectures such as ResNet-18,-34 and Mo-bileNetV2. For multi-bit quantization, UniQ outperforms existing methods to achieve the state-of-the-art accuracy. In binarization, the achieved accuracy is comparable to existing state-of-the-art methods even without modifying the original architectures.

show abstract

Section: Imagenet Resultsmentioning

confidence: 99%

“…Several studies increased the representation capacity by using more weight and activation bases [34,56]. Most studies incorporate changes to improve training efficiency such as dual skip connections [20,36,53]. In addition, more aggressive changes are sought for via neural architecture search [27,39,53].…”

Section: Introductionmentioning

confidence: 99%

Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer

2021

View full text Add to dashboard Cite

show abstract

“…The authors of all published works on BNN inference acceleration to date made use of high-precision floating-point data types during training (Courbariaux et al, 2015;Courbariaux & Bengio, 2016;Lin et al, 2017;Ghasemzadeh et al, 2018;Liu et al, 2018;Wang et al, 2019a;Umuroglu et al, 2020;He et al, 2020;Liu et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Enabling Binary Neural Network Training on the Edge

Wang

Davis

Moro

et al. 2021

Proceedings of the 5th International Workshop on Embedded and Mobile Deep Learning

View full text Add to dashboard Cite

“…For such edge devices, we propose compact architectures called Binarized LResNet18A (BLResNet18A) using Binary Neural Networks (BNNs) [27] where we binarize the filters, weights, biases, and activations. Different variants of BNNs [28]- [30] are proposed for use for traditional ML applications not only to avoid generalization error but also to achieve faster computation during deployment with one-bit xnor and bitcount operations [31]. But if the real network does not have enough complexity to extract the features, the regular BNN version suffers from poor representation power and has lower accuracy.…”

Section: Introductionmentioning

confidence: 99%

Binarized ResNet: Enabling Automatic Modulation Classification at the resource-constrained Edge

Deepsayan¹,

Priyadarshini²,

Nayak³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a ResNet based neural architecture to solve the problem of Automatic Modulation Classification. We showed that our architecture outperforms the stateof-the-art (SOTA) architectures. We further propose to binarize the network to deploy it in the Edge network where the devices are resource-constrained i.e. have limited memory and computing power. Instead of simple binarization, rotated binarization is applied to the network which helps to close the significant performance gap between the real and the binarized network. Because of the immense representation capability or the real network, its rotated binarized version achieves 85.33% accuracy compared to 95.76% accuracy of the proposed real network with 2.33 and 16 times lesser computing power than two of the SOTA architectures, MCNet and RMLResNet respectively, and approximately 16 times less memory than both. The performance can be improved further to 87.74% by taking an ensemble of four such rotated binarized networks.

show abstract

ProxyBNN: Learning Binarized Neural Networks via Proxy Matrices

Cited by 17 publications

References 25 publications

Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer

Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer

Enabling Binary Neural Network Training on the Edge

Binarized ResNet: Enabling Automatic Modulation Classification at the resource-constrained Edge

Contact Info

Product

Resources

About