EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

Abstract

In modern smart orchards, fruit detection models based on deep learning require expensive dataset labeling work to support the construction of detection models, resulting in high model application costs. Our previous work combined generative adversarial networks (GANs) and pseudolabeling methods to transfer labels from one specie to another to save labeling costs. However, only the color and texture features of images can be migrated, which still needs improvement in the accuracy of the data labeling. Therefore, this study proposes an EasyDAM_V2 model as an improved data labeling method for multishape and cross-species fruit detection. First, an image translation network named the Across-CycleGAN is proposed to generate fruit images from the source domain (fruit image with labels) to the target domain (fruit image without labels) even with partial shape differences. Then, a pseudolabel adaptive threshold selection strategy was designed to adjust the confidence threshold of the fruit detection model adaptively and dynamically update the pseudolabel to generate labels for images from the unlabeled target domain. In this paper, we use a labeled orange dataset as the source domain, and a pitaya, a mango dataset as the target domain, to evaluate the performance of the proposed method. The results showed that the average labeling precision values of the pitaya and mango datasets were 82.1% and 85.0%, respectively. Therefore, the proposed EasyDAM_V2 model is proven to be used for label transfer of cross-species fruit even with partial shape differences to reduce the cost of data labeling.

EasyDAM_V4: Guided-GAN-based cross-species data labeling for fruit detection with significant shape difference

“…For the fruit texture features, a texture loss function based on differentiable local binary pattern (LBP) descriptors is introduced in this paper. The fruit shape features follow the cross-loop multiscale structural similarity loss function describing the shape features that performed well in the previous study [ 8 ]. The color features are described jointly using cycle-consistent loss as well as identity loss.…”

“…In our previous study [ 8 ], the source domain fruit images were translated into simulated target domain fruit images by GANs and were constructed as a synthetic dataset. Then, we input to the anchor-free detector-based fruit detection model for feature extraction and training learning of fruit targets to obtain pseudo-labels of the target domain dataset.…”

“…The EasyDAM family of methods was proposed in previous research work. Among these methods, EasyDAM_V1 [ 7 ] focuses on the automatic labeling of fruit datasets with similar shapes, and EasyDAM_V2 [ 8 ] improves the generative adversarial network (GAN) model. Translation of fruit images with small shape differences can be performed, e.g.…”

“…However, the method still cannot solve the large deformation problem. Since the random selection of source domain fruit in EasyDAM_V1 [ 7 ] and EasyDAM_V2 [ 8 ] has the problem of unstable image generation quality of target domain fruit due to the selection difference, EasyDAM_V3 [ 9 ] proposes an optimal source domain establishment method based on a multidimensional spatial feature model to select the most suitable source domain, which can correspond to the multiple types of target domain fruit by one type of source domain fruit, making the model’s generalization greater. The pear is selected as the optimal source domain for experimental verification.…”

“…Some results have been achieved in the task of shape-changing image transformation. Our previous research work [ 8 ] compared the existing shape translation methods and proposed a method to calculate the structural consistency loss based on the cross-loop structure. By adding a multiscale fusion module this can realistically translate to get the target domain images with certain shape differences.…”