TaiJiNet: Towards Partial Binarized Convolutional Neural Network for Embedded Systems

Ling, Yingjian; Zhong, Kangping; Wu, Yingqin; Liu, Duo; Ren, Jing; Long, Linbo; Duan, Moming; Liu, Weichen; Liang, Liang

doi:10.1109/isvlsi.2018.00034

Cited by 5 publications

(5 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Low-rank decomposition [22], [23] Reduce the matrix size Effectively reduce storage and consumption,parameter is related to the number of network layers and is not effective for large-scale networks Pruning [24]- [26] Remove unimportant parameters High precision, less parameters, prevent over fitting,time-consuming and computational and the network is unstructured Quantization Convert floating point arithmetic to fixed point operation More concise model, difficult to implement , instability of accuracy, poor versatility Knowledge distillation [27], [28] [28] Transferring learning, training a small network Suitable for small model training ,artificial design seriously affects the training effect Binaryzation [12] Parameter binarization Greatly reduced storage,simple calculation, poor model performance function of the Jth channel of the output is shown in formula (1).…”

Section: Methods Description Advantages and Disadvantagesmentioning

confidence: 99%

“…Based on network compression, an algorithm compares the effects of precision and speed on different resolution weights, and it is shown that the lower the number of bits of the weight is, the lower the accuracy obtained. The network compresses the weight or activation value to one bit, and the compression ratio is 32:1 [10]- [12], [20]. [12] replaced some layers with the binary network layer.…”

Section: Introductionmentioning

confidence: 99%

“…The network compresses the weight or activation value to one bit, and the compression ratio is 32:1 [10]- [12], [20]. [12] replaced some layers with the binary network layer. Network compression dramatically improves the speed, while the accuracy is greatly affected.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High-Precision Binary Object Detector Based on a BSF-XNOR Convolutional Layer

Wang

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Recently, building an efficient and robust model for object detection has attracted the attention of the vision community. Although binary networks have a fast inference speed, they cannot be used directly on mobile devices such as unmanned aerial vehicles (UAVs) because of their low detection accuracy. Dfferent from improving the detection accuracy of a binary network by adjusting the network structure or adjusting the update gradient, we propose an improved binary neural network based on the block scaling factor XNOR (BSF-XNOR) convolutional layer. In addition, we propose a two-level densely connected network structure, which further enhances the network layer's feature representation capabilities. Experiments using the TensorFlow framework prove the effectiveness of our algorithm in improving accuracy. Compared with the original standard XNOR network, the mean average precision (mAP) detected by our algorithm on the PASCAL VOC dataset was improved. The experimental results on the VisDrone2019 UAV dataset confirm that our method achieves a better balance between speed and accuracy than previous methods. Our algorithm aims to guide and deploy high-precision binary networks on the embedded device and solves the problem of low-precision binary networks.

show abstract

Section: Methods Description Advantages and Disadvantagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High-Precision Binary Object Detector Based on a BSF-XNOR Convolutional Layer

Wang

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Most recently, hybrid quantization has attracted more and more attention, because it enables better trade-off between compression and performance [36]- [38]. As for partial binarization, a sub-area of hybrid quantization, on which we are focused, both training methods [39] and the corresponding hardware accelerators [19], [40] are also investigated extensively. The actual performance after compression heavily depends on the configuration of the partial binarization, i.e.…”

Section: A Cnn Compressionmentioning

confidence: 99%

“…Naive quantization usually leads to total failure especially for binarization. Significant effort has been devoted to develop better quantization and binarization methods as well as the hardware accelerator [14], [16]- [19]. Its success on CNNs has been demonstrated by multiple works, where memory consumption is deeply compressed although sometimes the performance cannot be preserved [20]- [23].…”

Section: Introductionmentioning

confidence: 99%

Binarizing Weights Wisely for Edge Intelligence: Guide for Partial Binarization of Deconvolution-Based Generators

Liu

Zhang

Ding

et al. 2020

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Automatic Group-Based Structured Pruning for Deep Convolutional Networks

Wei

Wang

Hua³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Structured pruning methods have been used in several convolutional neural networks (CNNs). However, group-based structured pruning is a challenging task. In previous methods, the number of groups is manually determined for all layers, which is suboptimal. Moreover, which kernels should be appropriately removed? Model accuracy may be significantly reduced when the number of kernels is removed. To address these challenges, we propose an automatic group-based structured pruning method with reinforcement learning, named AGSPRL, which can generate pruned models with different compression rates automatically. We first develop a reinforcement learning (RL) framework to learn the pruning rate for group-based channel pruning layer by layer. Then, based on the learned kernel pruning rate, we propose an efficient group configuration algorithm to adaptively determine the number of groups for each convolution layer. Finally, we introduce a channel pruning method with an attention mechanism as a tiny auxiliary filter selector for each group to dynamically determine which part of the kernels should be selected into the group convolution and which part of the kernels should be removed. To demonstrate the efficiency of our method, we apply it to a variety of CNNs in classification and detection datasets. The experimental results show that the AGSPRL not only adaptively but also accurately configures the number of groups. The accuracy is reduced by less than 1% and improved by 1%. Moreover, compared to other state-of-the-art methods, AGSPRL is more effective and has less accuracy loss.

show abstract

TaiJiNet: Towards Partial Binarized Convolutional Neural Network for Embedded Systems

Cited by 5 publications

References 3 publications

High-Precision Binary Object Detector Based on a BSF-XNOR Convolutional Layer

High-Precision Binary Object Detector Based on a BSF-XNOR Convolutional Layer

Binarizing Weights Wisely for Edge Intelligence: Guide for Partial Binarization of Deconvolution-Based Generators

Automatic Group-Based Structured Pruning for Deep Convolutional Networks

Contact Info

Product

Resources

About