Self-adversarial Training and Attention for Multi-task Wheat Phenotyping

Hu, Gensheng; Qian, Ling; Liang, Dong; Wan, Mingzhu

doi:10.13031/aea.13406

Cited by 15 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ResNet architecture employed comprises optimized convolutional layers, and a selfattention layer [37,38]. The self-attention mechanism helps model the relationship between distant regions of the input image, computing an attention weight matrix that indicates the most relevant regions for the prediction [39]. The hyperparameter search library Optuna (v2.3) [34] was used to determine the sp-DNN architecture: the number of layers, activation function, learning rate, and if attention layers were applied.…”

Section: Deep Learning Module For Multispectral Imagerymentioning

confidence: 99%

Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

et al. 2021

View full text Add to dashboard Cite

Assessing crop production in the field often requires breeders to wait until the end of the season to collect yield-related measurements, limiting the pace of the breeding cycle. Early prediction of crop performance can reduce this constraint by allowing breeders more time to focus on the highest-performing varieties. Here, we present a multimodal deep learning model for predicting the performance of maize (Zea mays) at an early developmental stage, offering the potential to accelerate crop breeding. We employed multispectral images and eight vegetation indices, collected by an uncrewed aerial vehicle approximately 60 days after sowing, over three consecutive growing cycles (2017, 2018 and 2019). The multimodal deep learning approach was used to integrate field management and genotype information with the multispectral data, providing context to the conditions that the plants experienced during the trial. Model performance was assessed using holdout data, in which the model accurately predicted the yield (RMSE 1.07 t/ha, a relative RMSE of 7.60% of 16 t/ha, and R2 score 0.73) and identified the majority of high-yielding varieties, outperforming previously published models for early yield prediction. The inclusion of vegetation indices was important for model performance, with a normalized difference vegetation index and green with normalized difference vegetation index contributing the most to model performance. The model provides a decision support tool, identifying promising lines early in the field trial.

show abstract

Section: Deep Learning Module For Multispectral Imagerymentioning

confidence: 99%

Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

et al. 2021

View full text Add to dashboard Cite

show abstract

“…By optimizing wheat seed detection, these authors achieved an error rate of less than 3% for the model. Hu et al [ 14 ] proposed a generative adversarial network based on an attention mechanism to count the number of wheat ears and spikelets, achieving 84.9% of the F1 value for identifying wheat ears and segmenting spikelets. Dandrifosse et al [ 15 ] used wheat images at the filling stage as the research object.…”

Section: Introductionmentioning

confidence: 99%

“…Although deep learning techniques have incomparable advantages in extracting wheat phenotypic information and achieving higher accuracy in image segmentation and recognition [ 17 ], acquiring images of wheat ears grain necessitate professional equipment such as CMOS cameras, which can be challenging to operate in complex production [ 18 ]. Furthermore, dense small targets pose challenging tasks for image recognition and segmentation because the adhesion between targets will likely occur, making accuracy improvement challenging [ 4 , 14 ].…”

Section: Introductionmentioning

confidence: 99%

Segmentation and counting of wheat spike grains based on deep learning and textural feature

Qian

Feng

et al. 2023

Plant Methods

View full text Add to dashboard Cite

Background Grain count is crucial to wheat yield composition and estimating yield parameters. However, traditional manual counting methods are time-consuming and labor-intensive. This study developed an advanced deep learning technique for the segmentation counting model of wheat grains. This model has been rigorously tested on three distinct wheat varieties: ‘Bainong 307’, ‘Xinmai 26’, and ‘Jimai 336’, and it has achieved unprecedented predictive counting accuracy. Method The images of wheat ears were taken with a smartphone at the late stage of wheat grain filling. We used image processing technology to preprocess and normalize the images to 480*480 pixels. A CBAM-HRNet wheat grain segmentation counting deep learning model based on the Convolutional Block Attention Module (CBAM) was constructed by combining deep learning, migration learning, and attention mechanism. Image processing algorithms and wheat grain texture features were used to build a grain counting and predictive counting model for wheat grains. Results The CBAM-HRNet model using the CBAM was the best for wheat grain segmentation. Its segmentation accuracy of 92.04%, the mean Intersection over Union (mIoU) of 85.21%, the category mean pixel accuracy (mPA) of 91.16%, and the recall rate of 91.16% demonstrate superior robustness compared to other models such as HRNet, PSPNet, DeeplabV3+ , and U-Net. Method I for spike count, which calculates twice the number of grains on one side of the spike to determine the total number of grains, demonstrates a coefficient of determination R2 of 0.85, a mean absolute error (MAE) of 1.53, and a mean relative error (MRE) of 2.91. In contrast, Method II for spike count involves summing the number of grains on both sides to determine the total number of grains, demonstrating a coefficient of determination R2 of 0.92, an MAE) of 1.15, and an MRE) of 2.09%. Conclusions Image segmentation algorithm of the CBAM-HRNet wheat spike grain is a powerful solution that uses the CBAM to segment wheat spike grains and obtain richer semantic information. This model can effectively address the challenges of small target image segmentation and under-fitting problems in training. Additionally, the spike grain counting model can quickly and accurately predict the grain count of wheat, providing algorithmic support for efficient and intelligent wheat yield estimation.

show abstract

“…Manual labeling is a heavy burden. Furthermore, some researchers applied adversarial learning to leaf and spikelet countings with unsupervised training (Giuffrida et al, 2019 ; Hu et al, 2019 ; Ayalew et al, 2020 ), but the models are difficult to train.…”

Section: Introductionmentioning

confidence: 99%

Automatic Detection and Counting of Wheat Spikelet Using Semi-Automatic Labeling and Deep Learning

Zhang

2022

Front. Plant Sci.

View full text Add to dashboard Cite

The number of wheat spikelets is an important phenotypic trait and can be used to assess the grain yield of the wheat crop. However, manual counting of spikelets is time-consuming and labor-intensive. To develop a cost-effective and highly efficient phenotyping system for counting the number of spikelets under laboratory conditions, methods based on imaging processing techniques and deep learning were proposed to accurately detect and count spikelets from color images of wheat spikes captured at the grain filling stage. An unsupervised learning-based method was first developed to automatically detect and label spikelets from spike color images and build the datasets for the model training. Based on the constructed datasets, a deep convolutional neural network model was retrained using transfer learning to detect the spikelets. Testing results showed that the root mean squared errors, relative root mean squared errors, and the coefficients of determination between the automatic and manual counted spikelets for four wheat lines were 0.62, 0.58, 0.54, and 0.77; 3.96, 3.73, 3.34, and 4.94%; and 0.73, 0.78, 0.84, and 0.67, respectively. We demonstrated that the proposed methods can effectively estimate the number of wheat spikelets, which improves the counting efficiency of wheat spikelets and contributes to the analysis of the developmental characteristics of wheat spikes.

show abstract

Self-adversarial Training and Attention for Multi-task Wheat Phenotyping

Cited by 15 publications

References 15 publications

Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

Segmentation and counting of wheat spike grains based on deep learning and textural feature

Automatic Detection and Counting of Wheat Spikelet Using Semi-Automatic Labeling and Deep Learning

Contact Info

Product

Resources

About