Semantic Segmentation of Agricultural Images Based on Style Transfer Using Conditional and Unconditional Generative Adversarial Networks

Abstract

Classification, segmentation, and recognition techniques based on deep-learning algorithms are used for smart farming. It is an important and challenging task to reduce the time, burden, and cost of annotation procedures for collected datasets from fields and crops that are changing in a wide variety of ways according to growing, weather patterns, and seasons. This study was conducted to generate crop image datasets for semantic segmentation based on an image style transfer using generative adversarial networks (GANs). To assess data-augmented performance and calculation burdens, our proposed framework comprises contrastive unpaired translation (CUT) for a conditional GAN, pix2pixHD for an unconditional GAN, and DeepLabV3+ for semantic segmentation. Using these networks, the proposed framework provides not only image generation for data augmentation, but also automatic labeling based on distinctive feature learning among domains. The Fréchet inception distance (FID) and mean intersection over union (mIoU) were used, respectively, as evaluation metrics for GANs and semantic segmentation. We used a public benchmark dataset and two original benchmark datasets to evaluate our framework of four image-augmentation types compared with the baseline without using GANs. The experimentally obtained results showed the efficacy of using augmented images, which we evaluated using FID and mIoU. The mIoU scores for the public benchmark dataset improved by 0.03 for the training subset, while remaining similar on the test subset. For the first original benchmark dataset, the mIoU scores improved by 0.01 for the test subset, while they dropped by 0.03 for the training subset. Finally, the mIoU scores for the second original benchmark dataset improved by 0.18 for the training subset and 0.03 for the test subset.

OutcropHyBNet: Hybrid Backbone Networks with Data Augmentation for Accurate Stratum Semantic Segmentation of Monocular Outcrop Images in Carbon Capture and Storage Applications

“…GANs offer a technique for generating realistic data, such as images, from random noise. In our previous study [34], we demonstrated the power and effectiveness of image synthesis for semantic segmentation applications in agriculture.…”

Oil Palm USB (Unstripped Bunch) Detector Trained on Synthetic Images Generated by PGGAN

“…Synthetic Datasets Goodfellow et al ( 2014) proposed GAN as a new generative modeling framework [14] to synthesize new data with the same characteristics from training examples, visually approximating the training data set. Various GANbased methods have been proposed for image synthesis in recent years [15], [16], [17], [18], [19], [20], [21], [22], [23], and [24] with applications spreading rapidly from computer vision and machine learning communities to domain-specific areas such as medical [25] [26], [27], [28], [29], and remote sensing [30], [31], [32] [33], [34], [35], [36], [37], [38], [39], [40], and [41]; industrial process [42], [43], [44], [45], [46], [47], and [48]; and agriculture [49], [50], [51], [52].…”

Crop and Weed Segmentation and Fractal Dimension Estimation Using Small Training Data in Heterogeneous Data Environment

“…Although the studies adopt similar concepts of fractal dimension (FD) estimations [12,13,15], 2 of 29 their applications are different in terms of building monitoring and medical image processing. Semantic segmentation also exhibits a fundamental role in the accurate recognition of crops and weeds [17,18]. Two mainstream methods exist for crop and weed detection.…”