Easy domain adaptation method for filling the species gap in deep learning-based fruit detection

Zhang, Wenli; Chen, Kaizhen; Wang, Jiaqi; Shi, Yun; Guo, Wei

doi:10.1038/s41438-021-00553-8

Cited by 26 publications

(35 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(1) Source domain orange dataset: the dataset follows the orange fruit dataset in the object detection dataset section of the EasyDAM [12] research. The orange images were mainly collected from an orange orchard in Sichuan (province), China.…”

Section: Fruit Image Translation Datasetsmentioning

confidence: 99%

EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

Zhang

Chen

et al. 2022

Plant Phenomics

Self Cite

View full text Add to dashboard Cite

In modern smart orchards, fruit detection models based on deep learning require expensive dataset labeling work to support the construction of detection models, resulting in high model application costs. Our previous work combined generative adversarial networks (GANs) and pseudolabeling methods to transfer labels from one specie to another to save labeling costs. However, only the color and texture features of images can be migrated, which still needs improvement in the accuracy of the data labeling. Therefore, this study proposes an EasyDAM_V2 model as an improved data labeling method for multishape and cross-species fruit detection. First, an image translation network named the Across-CycleGAN is proposed to generate fruit images from the source domain (fruit image with labels) to the target domain (fruit image without labels) even with partial shape differences. Then, a pseudolabel adaptive threshold selection strategy was designed to adjust the confidence threshold of the fruit detection model adaptively and dynamically update the pseudolabel to generate labels for images from the unlabeled target domain. In this paper, we use a labeled orange dataset as the source domain, and a pitaya, a mango dataset as the target domain, to evaluate the performance of the proposed method. The results showed that the average labeling precision values of the pitaya and mango datasets were 82.1% and 85.0%, respectively. Therefore, the proposed EasyDAM_V2 model is proven to be used for label transfer of cross-species fruit even with partial shape differences to reduce the cost of data labeling.

show abstract

Section: Fruit Image Translation Datasetsmentioning

confidence: 99%

EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

Zhang

Chen

et al. 2022

Plant Phenomics

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this context, expanding the coverage of a model trained in a domain to another domain without providing the training data set for the new domain, called domain adaptation, has been a hot topic in machine learning studies. Zhang et al (2021) proposed a domain adaptation method for fruit detection using a CNN model, CycleGAN ( Zhu et al 2017 ), based on GAN (Generative Adversarial Networks) ( Goodfellow et al 2014 ). CycleGAN is often used to transform images in a domain to those in another domain to learn the relationship between the two domains.…”

Section: Easing Training Data Provisions In Machine Learning Approachesmentioning

confidence: 99%

“…CycleGAN is often used to transform images in a domain to those in another domain to learn the relationship between the two domains. Zhang et al (2021) applied this feature of CycleGAN to automatically transform the training images manually annotated for orange fruit detection to the training images for fruits of other crops, such as apple and tomato fruits, without conducting the annotation process for those new crops. They trained a CycleGAN model to transform single orange images into single apple images, and the orange images of orange trees taken in an orchard were transformed into fake apple images using the trained CycleGAN.…”

Section: Easing Training Data Provisions In Machine Learning Approachesmentioning

confidence: 99%

High-throughput field crop phenotyping: current status and challenges

Ninomiya

2022

Breed. Sci.

View full text Add to dashboard Cite

In contrast to the rapid advances made in plant genotyping, plant phenotyping is considered a bottleneck in plant science. This has promoted high-throughput plant phenotyping (HTP) studies, resulting in an exponential increase in phenotyping-related publications. The development of HTP was originally intended for use as indoor HTP technologies for model plant species under controlled environments. However, this subsequently shifted to HTP for use in crops in fields. Although HTP in fields is much more difficult to conduct due to unstable environmental conditions compared to HTP in controlled environments, recent advances in HTP technology have allowed these difficulties to be overcome, allowing for rapid, efficient, non-destructive, noninvasive, quantitative, repeatable, and objective phenotyping. Recent HTP developments have been accelerated by the advances in data analysis, sensors, and robot technologies, including machine learning, image analysis, three dimensional (3D) reconstruction, image sensors, laser sensors, environmental sensors, and drones, along with high-speed computational resources. This article provides an overview of recent HTP technologies, focusing mainly on canopy-based phenotypes of major crops, such as canopy height, canopy coverage, canopy biomass, and canopy stressed appearance, in addition to crop organ detection and counting in the fields. Current topics in field HTP are also presented, followed by a discussion on the low rates of adoption of HTP in practical breeding programs.

show abstract

“…A major benefit of augmentations is their ability to generate data potentially modified for a large variety of conditions, enabling greater generalizability for agricultural models. Existing state-of-the-art work done for domain transfer often involves the usage of generative adversarial networks (GANs), such as Fei et al (2021), who developed a GAN for transferring sample imagery between day and night domains, and Zhang et al (2021), who used a CycleGAN network to edit the fruits present in imagery while maintaining the environmental conditions. While augmentations may not necessarily be able to provide an entire domain transfer as in the prior methods, they can still provide a generalization of conditions -for instance, rain and fog augmentations, as used in our research, can generalize data to a broader range of common conditions across the world.…”

Section: Augmentation Effectivenessmentioning

confidence: 99%

Standardizing and Centralizing Datasets to Enable Efficient Training of Agricultural Deep Learning Models

Joshi¹,

Guevara²,

Earles³

2022

Preprint

View full text Add to dashboard Cite

In recent years, deep learning models have become the standard for agricultural computer vision. Such models are typically fine-tuned to agricultural tasks using model weights that were originally fit to more general, non-agricultural datasets. This lack of agriculture-specific fine-tuning potentially increases training time and resource use, and decreases model performance, leading an overall decrease in data efficiency. To overcome this limitation, we collect a wide range of existing public datasets for three distinct tasks, standardize them, and construct standard training and evaluation pipelines, providing us with a set of benchmarks and pretrained models. We then conduct a number of experiments using methods which are commonly used in deep learning tasks, but unexplored in their domain-specific applications for agriculture. Our experiments guide us in developing a number of approaches to improve data efficiency when training agricultural deep learning models, without large-scale modifications to existing pipelines. Our results demonstrate that even slight training modifications, such as using agricultural pretrained model weights, or adopting specific spatial augmentations into data processing pipelines, can significantly boost model performance and result in shorter convergence time, saving training resources. Furthermore, we find that even models trained on low-quality annotations can produce comparable levels of performance to their high-quality equivalents, suggesting that datasets with poor annotations can still be used for training, expanding the pool of currently available datasets. Our methods are broadly applicable throughout agricultural deep learning, and present high potential for significant data efficiency improvements.

show abstract

Easy domain adaptation method for filling the species gap in deep learning-based fruit detection

Cited by 26 publications

References 40 publications

EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

High-throughput field crop phenotyping: current status and challenges

Standardizing and Centralizing Datasets to Enable Efficient Training of Agricultural Deep Learning Models

Contact Info

Product

Resources

About