Deep Neural Network Compression for Plant Disease Recognition

Wang, Ruiqing; Wu, Zongda; Ding, Jiuyang; Xia, Meng; Wang, Mengjian; Rao, Yuan; Jiang, Zhaohui

doi:10.3390/sym13101769

Cited by 9 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although many existing quantization methods can reduce the parameter storage of CNNs through encoding and decoding, the quantized parameters are still computed using floatingpoint numbers [23]. When implemented to the FPGA platform, the fixed-point number used differs in precision from the original floating-point number, which leads to calculation error and causes drop of inference accuracy.…”

Section: A Design Of Quantization Methodsmentioning

confidence: 99%

Research on Efficient CNN Acceleration Through Mixed Precision Quantization: A Comprehensive Methodology

He,

Liu,

Tahir

et al. 2023

IJACSA

View full text Add to dashboard Cite

To overcome challenges associated with deploying Convolutional Neural Networks (CNNs) on edge computing devices with limited memory and computing resources, we propose a mixed-precision CNN calculation method on a Field Programmable Gate Array (FPGA). This approach involves a collaborative design encompassing both software and hardware aspects. Initially, we devised a CNN quantization method tailored for the fixed-point operation characteristics of FPGA, addressing the computational challenges posed by floating-point parameters. We introduce a bit-width strategy search algorithm that assigns bit-widths to each layer based on CNN loss variation induced by quantization. Through retraining, this strategy mitigates the degradation in CNN inference accuracy. For FPGA acceleration design, we employ a flow processing architecture with multiple Processing Elements (PEs) to support mixed-precision CNNs. Our approach incorporates a folding design method to implement shared PEs between layers, significantly reducing FPGA resource usage. Furthermore, we designed a data reading method, incorporating a register set buffer between memory and processing elements to alleviate issues related to mismatched data reading and computing speeds. Our implementation of the mixed-precision ResNet20 model on the Kintex-7 Eco R2 development board achieves an inference accuracy of 91.68% and a computing speed 4.27 times faster than the Central Processing Unit (CPU) on the CIFAR-10 dataset, with an accuracy drop of only 1.21%. Compared to a unified 16-bit FPGA accelerator design method, our proposed approach demonstrates an 89-fold increase in computing speed while maintaining similar accuracy.

show abstract

Section: A Design Of Quantization Methodsmentioning

confidence: 99%

Research on Efficient CNN Acceleration Through Mixed Precision Quantization: A Comprehensive Methodology

He,

Liu,

Tahir

et al. 2023

IJACSA

View full text Add to dashboard Cite

show abstract

“…With the continuous improvement in computer processing power, deep learning has made remarkable progress in various fields [1]- [3]. In the agricultural domain, deep models have demonstrated outstanding performance in areas such as plant diseases identification [4], fruit counting [5], as well as other applications [6][7]. Intelligent tea picking is also a highly researched direction in agricultural applications.…”

Section: Introductionmentioning

confidence: 99%

Learning Lightweight Tea Detector with Reconstructed Feature and Dual Distillation

Zheng,

Zuo,

Zhang

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Currently, image recognition based on deep neural networks has become the mainstream direction of research, and significant progress has been made in its application in the field of tea detection. Many deep models exhibit high recognition rates in tea leaves detection. However, deploying these models directly on tea-picking equipment in natural environments is impractical. The extremely high parameters and computational complexity of these models make it challenging to perform real-time tea leaves detection. Meanwhile, lightweight models struggle to achieve competitive detection accuracy. Therefore, this paper addresses the issue of computational resource constraints in remote mountain areas and proposes Reconstructed Feature and Dual Distillation (RFDD) to enhance the detection capability of lightweight models for tea leaves. In our method, the Reconstructed Feature selectively masks the feature of the student model based on the spatial attention map of the teacher model and utilizes a generation block to force the student model to generate the teacher’s full feature. The Dual Distillation comprises Decoupled Distillation and Global Distillation. Decoupled Distillation divides the reconstructed feature into foreground and background features based on the Ground-Truth. This compels the student model to allocate different attention to foreground and background, focusing on their critical pixels and channels. However, Decoupled Distillation leads to the loss of relation knowledge between foreground and background pixels. Therefore, we further perform Global Distillation to extract this lost knowledge. Since RFDD only requires loss calculation on feature map, it can be easily applied to various detectors. We conducted experiments on detectors with different frameworks, using a tea dataset captured at the Huangshan Houkui Tea Plantation. The experimental results indicate that, under the guidance of RFDD, the student detectors have achieved performance improvements to varying degrees. For instance, a one-stage detector like RetinaNet (ResNet-50) experienced a 3.14% increase in Average Precision (AP) after RFDD guidance. Similarly, a two-stage model like Faster RCNN (ResNet-50) obtained a 3.53% improvement in AP. This offers promising prospects for lightweight models to efficiently perform real-time tea leaves detection tasks.

show abstract

“…However, using knowledge distillation only after the pruning has been completed, and not while it is in progress, may result in suboptimal model performance. In addition, most of the previous studies have focused on the problem of how to improve the performance of unstructured pruning [13,14], while there have been few studies on structured pruning [15,16]. In fact, non-structured pruning needs special software libraries or hardware to speed up the network model, whereas structured pruning can compress the network without any help [17].…”

Section: Introductionmentioning

confidence: 99%

Progressive multi-level distillation learning for pruning network

Wang

Wan

et al. 2023

Complex Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Although the classification method based on the deep neural network has achieved excellent results in classification tasks, it is difficult to apply to real-time scenarios because of high memory footprints and prohibitive inference times. Compared to unstructured pruning, structured pruning techniques can reduce the computation cost of the model runtime more effectively, but inevitably reduces the precision of the model. Traditional methods use fine tuning to restore model damage performance. However, there is still a large gap between the pruned model and the original one. In this paper, we use progressive multi-level distillation learning to compensate for the loss caused by pruning. Pre-pruning and post-pruning networks serve as the teacher and student networks. The proposed approach utilizes the complementary properties of structured pruning and knowledge distillation, which allows the pruned network to learn the intermediate and output representations of the teacher network, thus reducing the influence of the model subject to pruning. Experiments demonstrate that our approach performs better on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets with different pruning rates. For instance, GoogLeNet can achieve near lossless pruning on the CIFAR-10 dataset with 60% pruning. Moreover, this paper also proves that using the proposed distillation learning method during the pruning process achieves more significant performance gains than after completing the pruning.

show abstract

Deep Neural Network Compression for Plant Disease Recognition

Cited by 9 publications

References 31 publications

Research on Efficient CNN Acceleration Through Mixed Precision Quantization: A Comprehensive Methodology

Research on Efficient CNN Acceleration Through Mixed Precision Quantization: A Comprehensive Methodology

Learning Lightweight Tea Detector with Reconstructed Feature and Dual Distillation

Progressive multi-level distillation learning for pruning network

Contact Info

Product

Resources

About