Transfer Learning for Leaf Small Dataset Using Improved ResNet50 Network with Mixed Activation Functions

Abstract

Taxonomic studies of leaves are one of the most effective means of correctly identifying plant species. In this paper, mixed activation function is used to improve the ResNet50 network in order to further improve the accuracy of leaf recognition. Firstly, leaf images of 15 common tree species in northern China were collected from the Urban Forestry Demonstration Base of Northeast Forestry University (45°43′–45°44′ N, 126°37′–126°38′ E, forest type was artificial forest), and a small leaf dataset was established. After that, seven commonly used activation functions were selected to improve the ResNet50 network structure, and the improved network was applied to the transfer learning research of the leaf small dataset. On this basis, five activation functions with better performance were selected for the study of mixed activation functions in deep learning. Two of these five activation functions are arbitrarily selected for combination, and a total of twenty combinations are obtained. Further, the first activation function was used in each combination to replace the first ReLU function after all addition operations in the ResNet50 network residual block structure, and another activation function was used to replace the other position ReLU functions. The experimental results show that in the transfer learning of the leaf small dataset using the ResNet50 deep residual network, the appropriate combination of mixed activation functions can increase the performance of the improved network to a certain extent. Among them, the ELU-Swish1 combination has the most significant improvement effect on the network performance, whose final effective validation accuracy reaches 98.17%. Furthermore, the comparison with GoogLeNet and VGG-16 also demonstrates the excellent performance of the improved ELU-Swish1 ResNet50 (ES-ResNet50) network architecture. Finally, tests on the other two small leaf datasets, Flavia and Swedish, also demonstrate the performance improvement of ES-ResNet50. The validation accuracy of the improved ES-Resnet 50 algorithm on these two datasets reaches 99.30% and 99.39%, respectively. All these experiments prove that the recognition performance of leaf transfer learning using the ES-ResNet50 network is indeed improved, which may be caused by the complementarity of the e-exponential gradient of ELU and Swish1 activation functions in the negative region.

Convolutional neural network for estimation of harvest time of forage sorghum (sorghum bicolor) cultivar samurai-1

“…The pre-processed dataset will be a source for input into each CNN algorithm with various architectures, shown in Figure 4. There is the CNN algorithm with simple architectures [18], the CNN algorithm with a residual network-50 (ResNet50) architecture [19], [20], the CNN algorithm with a visual geometry group (VGG) 16 architecture [21], the CNN algorithm with convolutional neural networks for mobile vision applications (MobileNet) architecture [22] and CNN algorithm with an inception architecture [23]. Comparison graphically, the visualization CNN model architectures are simple architectures shown in Figure 4…”

Secure Convolutional Neural Network-Based Internet-of-Healthcare Applications

“…IV-A). As our first contribution is to provide a comparison between model complexity and its effect on robustness against attacks, we use five well-known pre-trained networks, such as LeNet5 [65], MobileNetV1 [66], VGG16 [67], ResNet50 [68], and InceptionV3 [69]. The first represents the model with the lowest computational cost compared to the others.…”

Dual-Channel Semi-Supervised Adversarial Network for Building Segmentation from UAV-Captured Images

“…Figure 3 shows the architectural design of the Deeplabv3+ model, a building segmentation approach, with ResNet50 as the primary backbone, based on prior research for feature extraction [39][40][41].…”