Style-Consistent Image Translation: A Novel Data Augmentation Paradigm to Improve Plant Disease Recognition

Abstract

Deep learning shows its advantages and potentials in plant disease recognition and has witnessed a profound development in recent years. To obtain a competing performance with a deep learning algorithm, enough amount of annotated data is requested but in the natural world, scarce or imbalanced data are common, and annotated data is expensive or hard to collect. Data augmentation, aiming to create variations for training data, has shown its power for this issue. But there are still two challenges: creating more desirable variations for scarce and imbalanced data, and designing a data augmentation to ease object detection and instance segmentation. First, current algorithms made variations only inside one specific class, but more desirable variations can further promote performance. To address this issue, we propose a novel data augmentation paradigm that can adapt variations from one class to another. In the novel paradigm, an image in the source domain is translated into the target domain, while the variations unrelated to the domain are maintained. For example, an image with a healthy tomato leaf is translated into a powdery mildew image but the variations of the healthy leaf are maintained and transferred into the powdery mildew class, such as types of tomato leaf, sizes, and viewpoints. Second, current data augmentation is suitable to promote the image classification model but may not be appropriate to alleviate object detection and instance segmentation model, mainly because the necessary annotations can not be obtained. In this study, we leverage a prior mask as input to tell the area we are interested in and reuse the original annotations. In this way, our proposed algorithm can be utilized to do the three tasks simultaneously. Further, We collect 1,258 images of tomato leaves with 1,429 instance segmentation annotations as there is more than one instance in one single image, including five diseases and healthy leaves. Extensive experimental results on the collected images validate that our new data augmentation algorithm makes useful variations and contributes to improving performance for diverse deep learning-based methods.

Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review

“…Liu et al (2020) presented a GAN model with a channel decreasing generator to synthesize 4-class grape leaf images, reporting 98.7% classification accuracy, which is about 3% and 2% better than the models without and with only basic image augmentation, respectively. Recently, Xu et al (2022) adapted style consistent image translation GAN (SCITGAN) for tomato disease recognition. A five-class tomato leaf dataset was collected for GAN training to generate images (Fig.…”

“…These studies provide overwhelming evidence that DL models substantially benefit from GAN-augmented data. For instance, Xu et al (2022) observed the plant disease recognition rate increasing from 86.75% of the original data to 94.61% based on the ensemble of the original, GANsynthesized images and the data from traditional augmentation. Abbas et al (2021) obtained 97.11% accuracy on GAN-augmented images compared to 94.34% on original.…”

“…As discussed above, GAN models have the risk of generating low-quality images, while this may not be a real problem for traditional data augmentation methods that generally do not significantly impair image quality. That said, GAN-based data augmentation may not always translate into performance gains of DL models compared to traditional data augmentation or using the original data, which is observed in many application studies (Xu et al, 2022;Hartley and French, 2021;Douarre et al, 2019;Fawakherji et al, 2021). However, strong evidence shows consistent improvements can be obtained by using both GAN and traditional augmentation than performing them separately, or by mixing the original and GANaugmented data.…”

A Comprehensive Survey of Image Augmentation Techniques for Deep Learning

“…However, collecting images is expensive and hard in many cases. Thus, obtaining comparable performance with a limited number of images is one issue, especially in practical applications, such as medical [1] and agricultural images [2].…”

Automatic Plant Disease Detection Based on Tranvolution Detection Network With GAN Modules Using Leaf Images

“…Liu and Wang ( 2020 ) improved the existing technique of tomato pest image recognition based on the YOLO-v3 model (an efficient object detection algorithm based on CNNs), improve the existing technique of tomato pest image recognition in the natural. Xu et al ( 2022 ) provided an approach for data augmentation that can fully utilize data from non-target regions of sample images to optimize deep learning models for disease detection. Their method is more applicable to plant disease detection than common data enhancement approaches.…”