SpikeSegNet-a deep learning approach utilizing encoder-decoder network with hourglass for spike segmentation and counting in wheat plant from visual imaging

Abstract

Background: High throughput non-destructive phenotyping is emerging as a significant approach for phenotyping germplasm and breeding populations for the identification of superior donors, elite lines, and QTLs. Detection and counting of spikes, the grain bearing organs of wheat, is critical for phenomics of a large set of germplasm and breeding lines in controlled and field conditions. It is also required for precision agriculture where the application of nitrogen, water, and other inputs at this critical stage is necessary. Further, counting of spikes is an important measure to determine yield. Digital image analysis and machine learning techniques play an essential role in non-destructive plant phenotyping analysis. Results:In this study, an approach based on computer vision, particularly object detection, to recognize and count the number of spikes of the wheat plant from the digital images is proposed. For spike identification, a novel deeplearning network, SpikeSegNet, has been developed by combining two proposed feature networks: Local Patch extraction Network (LPNet) and Global Mask refinement Network (GMRNet). In LPNet, the contextual and spatial features are learned at the local patch level. The output of LPNet is a segmented mask image, which is further refined at the global level using GMRNet. Visual (RGB) images of 200 wheat plants were captured using LemnaTec imaging system installed at Nanaji Deshmukh Plant Phenomics Centre, ICAR-IARI, New Delhi. The precision, accuracy, and robustness (F 1 score) of the proposed approach for spike segmentation are found to be 99.93%, 99.91%, and 99.91%, respectively. For counting the number of spikes, "analyse particles"-function of imageJ was applied on the output image of the proposed SpikeSegNet model. For spike counting, the average precision, accuracy, and robustness are 99%, 95%, and 97%, respectively. SpikeSegNet approach is tested for robustness with illuminated image dataset, and no significant difference is observed in the segmentation performance. © The Author(s) 2020. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article' s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article' Conclusion: In this study, a new approach called as SpikeSegNet has been proposed based on combined digital image analysis and deep learning techniques. A dedicated deep learning approach has been developed to identify and count spikes in the wheat plants. The performance of the approach demonstrates that SpikeSegNet is an effective and robust approach for spike detection and counting. As detection and counting of wheat spikes are cl...

Web-SpikeSegNet: Deep Learning Framework for Recognition and Counting of Spikes from Visual Images of Wheat Plants

“…Web SpikeSegNet is developed based on the approach give by Misra et al (2020) [3]. The approch is based on convolutional encoder-decoder deep-learning technique for pixel-wise segmentation of spikes from the wheat plant's visual images.…”

“…SpikeSegNet consists of two modules viz., Local Patch extraction Network (LPNet) and Global Mask Refinement Network (GMRNet), in sequential order. The details of the approach are given in [3]. Input images were divided into patches before entering into the LPNet module to facilitate local features' learning more effectively than the whole input image.…”

“…Results: In the present study, we developed an online platform "Web-SpikeSegNet" based on a deep-learning framework for spike detection and counting from the wheat plant's visual images. This platform implements the "SpikeSegNet" approach developed by Misra et al(2020), which has proved as an effective and robust approach for spike detection and counting. Architecture of the Web-SpikeSegNet consists of 2 layers.…”

“…Presently, non-destructive image analysis based phenotyping is gaining momentum and proved as less laborious and fast method. A cluster of research works available in the area of computer vision to detect and characterize spikes and spikelets in wheat plants [3,4,5,6,7,8]. In computer vision, the problem of spike detection lies under the domain of pixel-wise segmentation of object.…”

“…Pound et al (2017) [6] and Hasan et al (2018) [8] used Autoencoder [9] and Region-based Convolutional Neural Network (R-CNN) [9] deep-learning technique, respectively, to detect and characterize spikes with greater than 90 percent accuracy. Recently, Misra et al(2020) [3] developed a deep learning model known as SpikeSegNet, which was reported as an effective and robust approach for spike detection (accuracy: 99.91 percent) and counting (accuracy: 95 percent) from visual images irrespective of various illumination factors. In this paper, a web-solution is presented as "Web-SpikeSegNet" for spike segmentation and counting from visual images of wheat plants for easy accessibility and quick reference.…”