End-to-end 3D CNN for plot-scale soybean yield prediction using multitemporal UAV-based RGB images

Abstract

Crop yield prediction from UAV images has significant potential in accelerating and revolutionizing crop breeding pipelines. Although convolutional neural networks (CNN) provide easy, accurate and efficient solutions over traditional machine learning models in computer vision applications, a CNN training requires large number of ground truth data, which is often difficult to collect in the agricultural context. The major objective of this study was to develope an end-to-end 3D CNN model for plot-scale soybean yield prediction using multitemporal UAV-based RGB images with approximately 30,000 sample plots. A low-cost UAV-RGB system was utilized and multitemporal images from 13 different experimental fields were collected at Argentina in 2021. Three commonly used 2D CNN architectures (i.e., VGG, ResNet and DenseNet) were transformed into 3D variants to incorporate the temporal data as the third dimension. Additionally, multiple spatiotemporal resolutions were considered as data input and the CNN architectures were trained with different combinations of input shapes. The results reveal that: (a) DenseNet provided the most efficient result (R2 0.69) in terms of accuracy and model complexity, followed by VGG (R2 0.70) and ResNet (R2 0.65); (b) Finer spatiotemporal resolution did not necessarily improve the model performance but increased the model complexity, while the coarser resolution achieved comparable results; and (c) DenseNet showed lower clustering patterns in its prediction maps compared to the other models. This study clearly identifies that multitemporal observation with UAV-based RGB images provides enough information for the 3D CNN architectures to accurately estimate soybean yield non-destructively and efficiently.

Review of Crop Phenotyping in Field Plot Experiments Using UAV-Mounted Sensors and Algorithms

“…However, researchers should note that the more complexity increases in data collection and model architecture, the more it costs the practitioners to implement. A higher model complexity and finer spatiotemporal resolution of UAV remote sensing did not necessarily improve crop yield prediction accuracy in deep learning models [77,82]. Furthermore, although deep learning models outperformed traditional statistical and machine learning models, the degree of model performance improvements is sometimes not substantial, and the ultimate model performance generally scored at the moderate level.…”

“…Another study on wheat yield prediction indicated that although the RMSE values decreased by approximately 0.06 t ha −1 in the CNN model compared to the linear regression model based on the vegetation index (i.e., enhanced vegetation index 2), the degree of the prediction accuracy improvement in the CNN was not substantial [77]. The most recent work developed soybean yield prediction using multitemporal UAV-based RGB images and more complicated CNN models, but the most efficient model showed a moderate yield prediction accuracy (R 2 = 0.6) [82]. A CNN generally requires hundreds or thousands of training datasets, which may hinder practical phenotyping applications.…”

An Overview of Machine Learning Applications on Plant Phenotyping, with a Focus on Sunflower

“…Crop management can also use CNNs trained to find patterns starting with sample images. A 3D CNN was used to predict soybean yield via analysing multitemporal images [141].…”

Explainable AI Based framework for Banana Disease Detection

“…Artificial Intelligence(AI) contributed to various field such as biomedical [3][4], agriculture, education and fintech [2][5][6] [7]. There is a Tremendous interest towards aggrotech field in terms of automated farm management, drone for field inspection followed by spray of pesticide [1] [8]. Disease detection is an important stage in consideration of plant life cycle mostly diseases noticed in the stem and leaf, it is a challenging task for individuals.…”

“…Global food insecurity is raising concern for humanity due to raise in global warming and population led to a demand supply gap in food production [1]. In terms of food security in recent days playing an important role in addressing the root causes associated with yield [2].…”