A Sparse Deep Transfer Learning Model and Its Application for Smart Agriculture

Abstract

The introduction of deep transfer learning (DTL) further reduces the requirement of data and expert knowledge in various uses of applications, helping DNN-based models effectively reuse information. However, it often transfers all parameters from the source network that might be useful to the task. The redundant trainable parameters restrict DTL in low-computing-power devices and edge computing, while small effective networks with fewer parameters have difficulty transferring knowledge due to structural differences in design. For the challenge of how to transfer a simplified model from a complex network, in this paper, an algorithm is proposed to realize a sparse DTL, which only transfers and retains the most necessary structure to reduce the parameters of the final model. Sparse transfer hypothesis is introduced, in which a compressing strategy is designed to construct deep sparse networks that distill useful information in the auxiliary domain, improving the transfer efficiency. The proposed method is evaluated on representative datasets and applied for smart agriculture to train deep identification models that can effectively detect new pests using few data samples.

Tiny machine learning on the edge: A framework for transfer learning empowered unmanned aerial vehicle assisted smart farming

“…Much research effort has also addressed the use of TL in smart agriculture applications showing the promising advantages of the approach in enhancing the agriculture process. These efforts present the concept, tools, advantages and application of TL in smart agriculture [32][33][34][35][36][37]. In our work, we focus on implementing ML on the no-OS chips that run on ultra-low performance capabilities.…”