High-Throughput Phenotyping of Soybean Maturity Using Time Series UAV Imagery and Convolutional Neural Networks

Abstract

Soybean maturity is a trait of critical importance for the development of new soybean cultivars, nevertheless, its characterization based on visual ratings has many challenges. Unmanned aerial vehicles (UAVs) imagery-based high-throughput phenotyping methodologies have been proposed as an alternative to the traditional visual ratings of pod senescence. However, the lack of scalable and accurate methods to extract the desired information from the images remains a significant bottleneck in breeding programs. The objective of this study was to develop an image-based high-throughput phenotyping system for evaluating soybean maturity in breeding programs. Images were acquired twice a week, starting when the earlier lines began maturation until the latest ones were mature. Two complementary convolutional neural networks (CNN) were developed to predict the maturity date. The first using a single date and the second using the five best image dates identified by the first model. The proposed CNN architecture was validated using more than 15,000 ground truth observations from five trials, including data from three growing seasons and two countries. The trained model showed good generalization capability with a root mean squared error lower than two days in four out of five trials. Four methods of estimating prediction uncertainty showed potential at identifying different sources of errors in the maturity date predictions. The architecture developed solves limitations of previous research and can be used at scale in commercial breeding programs.

Response of a Diverse European Soybean Collection to “Short Duration” and “Long Duration” Drought Stress

“…Manual measurements and visual observations are still broadly applied, although interest in the use of unmanned aerial vehicles (UAV) equipped with different imaging sensors for soybean phenotyping is on the rise. UAVs have been employed for quantification of wilting ( Zhou et al, 2020 ), estimation of maturity stage ( Yu et al, 2016 ; Zhou et al, 2019 ; Trevisan et al, 2020 ), quantification of plant density ( Ranđelović et al, 2020 ) or leaf area index ( Yuan et al, 2017 ), and prediction of yield ( Yu et al, 2016 ; Zhang et al, 2019b ; Herrero-Huerta et al, 2020 ; Maimaitijiang et al, 2020 ; Zhou et al, 2021 ). Similarly, in previous work ( Borra-Serrano et al, 2020 ) we developed a UAV-based approach to estimate canopy cover and canopy height and to derive parameters related to growth and development in soybean.…”

Classification of Soybean Pubescence from Multispectral Aerial Imagery

“…Several options exist which could be tested, including spatially aware pixel classifiers, which would use information from neighbouring pixels to help improve classification accuracy [ 33 ]. Another option would be to use convolutional neural nets (CNNs) where entire plot images are used for classification, which was recently used for estimating soybean maturity [ 6 ]. However, the training data requirements for such a model are much larger than the SVM classifier used here.…”

“…Aerial imagery has been successfully deployed to assess a number of traits in soybean, with publications becoming more frequent in recent years with the decrease in price for imaging platforms and increasing availability of analytical methods to handle the data for plant researchers. Determination of soybean maturity has been effectively demonstrated from aerial imagery using several methods: partial least squares regression [ 4 ], random forest supervised machine learning [ 5 ], and convolutional neural networks [ 6 ]. Yield predictions from time-course imagery show promise for estimating final yield [ 5 ], and further work using deep neural nets continues this trend [ 7 ].…”

The numbers game of soybean breeding in the United States

“…For instance, plant breeders may use molecular data to reduce segregation among traits (keep or increase desired fixed alleles) of interest among the crossing parents while simultaneously promoting recombination to achieve novel allelic combinations. In addition, aerial-based robotic platforms equipped with multiple sensors can rapidly and non-destructively collect an extensive amount of data which, when integrated with advanced computer vision, artificial intelligence, and big data analytics, can estimate phenotypes that are otherwise labor and costintensive including plant maturity (Zhou et al 2019, Trevisan et al 2020, plant height (Zhou et al 2021), canopy coverage (Moreira et al 2019), yield performance (Herrero-Huerta et al 2020, Maimaitijiang et al 2020, Zhou et al 2021, and biotic and abiotic tolerance (Zhou et al 2021). In the early stages (such as the progeny row stage) of a breeding pipeline where representative yield observations are limited and/or not possible to be obtained, the adoption of phenomics can assist breeders to make decisions with more confidence and quickly advance genotypes throughout the pipeline (Moreira et al 2020).…”