Aerial Imagery Analysis – Quantifying Appearance and Number of Sorghum Heads for Applications in Breeding and Agronomy

Abstract

Sorghum (Sorghum bicolor L. Moench) is a C4 tropical grass that plays an essential role in providing nutrition to humans and livestock, particularly in marginal rainfall environments. The timing of head development and the number of heads per unit area are key adaptation traits to consider in agronomy and breeding but are time consuming and labor intensive to measure. We propose a two-step machine-based image processing method to detect and count the number of heads from high-resolution images captured by unmanned aerial vehicles (UAVs) in a breeding trial. To demonstrate the performance of the proposed method, 52 images were manually labeled; the precision and recall of head detection were 0.87 and 0.98, respectively, and the coefficient of determination (R2) between the manual and new methods of counting was 0.84. To verify the utility of the method in breeding programs, a geolocation-based plot segmentation method was applied to pre-processed ortho-mosaic images to extract >1000 plots from original RGB images. Forty of these plots were randomly selected and labeled manually; the precision and recall of detection were 0.82 and 0.98, respectively, and the coefficient of determination between manual and algorithm counting was 0.56, with the major source of error being related to the morphology of plants resulting in heads being displayed both within and outside the plot in which the plants were sown, i.e., being allocated to a neighboring plot. Finally, the potential applications in yield estimation from UAV-based imagery from agronomy experiments and scouting of production fields are also discussed.

A Deep Learning Semantic Segmentation-Based Approach for Field-Level Sorghum Panicle Counting

“…With the development of deep learning models for point clouds [57,58], compared performance should be achieved from point clouds. Thought not directly comparable, due to the use of different accuracy metrics, the performance achieved here is generally in line to other previous studies that applied image data [34,37,38]. In Olsen et al [34] a 98% counting accuracy and median absolute error (MAE) between 1.88 and 2.66 were achieved for 18 sorghum varieties in the Midwestern United States.…”

“…The accuracy was based on a receiver operating characteristic (ROC) curve's area under the curve (AUC) metric-which is usually correlated with overall accuracy. Similarly performance was also reported in [37]. As highlighted before, this high accuracy came at the expense of significant feature engineering, which we circumvented in this study by applying a deep learning approach.…”

“…While high accuracy was achievable, it came at the expense of significant feature engineering. A similar approach based on learning multiple features for panicle detection is presented in [37]. To reduce the effort spent on manual labeling, Ghosal et al [38] applied a weakly supervised deep learning framework whereby a model was trained on a small training sample but gradually updated through human feedback on object detection results.…”

Active learning with point supervision for cost-effective panicle detection in cereal crops

“…State-of-the-art convolutional neural networks have been shown to perform well on a wide variety of phenotyping tasks. The applications of CNNs in plant phenotyping include image classification tasks such as plant species identification [1], stress identification [2], object detection arXiv:1910.01789v2 [cs.CV] 15 Oct 2019 and counting tasks such as panicle or spike detection [3,4,5,6], leaf counting [7], fruit detection [8]; as well as pixel-wise segmentation based tasks such as panicle segmentation [9,10] and crop-weed segmentation [11]. We refer the reader to [12] and [13] for a full treatment of deep learning in agriculture and plant phenotyping tasks.…”

Learning from Synthetic Dataset for Crop Seed Instance Segmentation

“…This order of annotated dataset is generally difficult to prepare for an individual or a research group. In the agricultural domain, it has been reported that sorghum head detection network can be trained with a dataset consisting of 52 images with an average of 400 objects per image 16 , while a crop stem detection network was trained starting from 822 images 17 . These case studies imply that the amount of data required in a specialized task may be less compared to a relatively generalized task such as ImageNet classification and COCO detection challenges.…”