Learning from Synthetic Dataset for Crop Seed Instance Segmentation

Abstract

12 13 *Correspondence should be addressed to Yosuke Toda; tyosuke@aquaseerser.com 14 15Incorporating deep learning in the image analysis pipeline has opened the possibility of introducing precision 16 phenotyping in the field of agriculture. However, to train the neural network, a sufficient amount of training 17 data must be prepared, which requires a time-consuming manual data annotation process that often becomes 18 the limiting step. Here, we show that an instance segmentation neural network (Mask R-CNN) aimed to 19 phenotype the barley seed morphology of various cultivars, can be sufficiently trained purely by a 20 synthetically generated dataset. Our attempt is based on the concept of domain randomization, where a large 21 amount of image is generated by randomly orienting the seed object to a virtual canvas. After training with 22 such a dataset, performance based on recall and the average Precision of the real-world test dataset achieved 23 96% and 95%, respectively. Applying our pipeline enables extraction of morphological parameters at a large 24 scale, enabling precise characterization of the natural variation of barley from a multivariate perspective. 25Importantly, we show that our approach is effective not only for barley seeds but also for various crops 26 including rice, lettuce, oat, and wheat, and thus supporting the fact that the performance benefits of this 27 technique is generic. We propose that constructing and utilizing such synthetic data can be a powerful method 28 to alleviate human labor costs needed to prepare the training dataset for deep learning in the agricultural 29 domain. 30 31 32

Reference Measurements in Developing UAV Systems for Detecting Pests, Weeds, and Diseases

“…However, recent advances in the computer vision domain led to the emergence of several studies that attempt to bridge the sim2real reality gap. These studies either train convolutional neural networks only on synthetically generated data [113] or combine training on synthetic and manually labelled reference data [114,115]. Studies that combine synthetically generated data with manually labelled reference data have shown promising results [115].…”

“…In the context of agriculture, the authors in [116] generated synthetic images of Arabidopsis (small flowering plants) from 3D models that can potentially accelerate the field of plant phenotyping [116]. The study in [113] generated synthetic data for different seeds, including barley and wheat, against a black background; the goal was to detect and segment each seed in the pictures. The manual labelling of each seed would take a long time compared to creating the reference data using simulation.…”

“…The manual labelling of each seed would take a long time compared to creating the reference data using simulation. The authors claimed a high accuracy and concluded that the same approach could be extended to other crops like rice, oats, and lettuce [113].…”

Recent Data Augmentation Strategies for Deep Learning in Plant Phenotyping and Their Significance

“…The replication of this technique in agricultural phenotyping has also been attested in crop seed segmentation. Presented in [18], the cut and paste technique showed to be useful and to generalise to many types of seeds in segmentation tasks. The synthetic dataset was created by randomly rotating and pasting seed instances into background extracted from the real images.…”

“…Examples of the synthetic images of seeds from different species generated by the method presented in[18].…”