MineGAN: Effective Knowledge Transfer From GANs to Target Domains With Few Images

Wang, Yaxing; González-García, Abel; Berga, David; Herranz, Luis; Khan, Fahad Shahbaz; Weijer, Joost van de

doi:10.1109/cvpr42600.2020.00935

Cited by 145 publications

(131 citation statements)

References 13 publications

Supporting

Mentioning

126

Contrasting

Order By: Relevance

“…Fine-tuning provides training-quality visual feedback more quickly and thus offers a more efficient way to learn model behavior and hyperparameters. It has been reported in GANs-related studies [48,49] that transfer learning is beneficial in saving a large amount of training time and computation resources. In this study, we adopted the transfer learning in all the experiments.…”

Section: Improved Strategymentioning

confidence: 99%

Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier

et al. 2021

View full text Add to dashboard Cite

Automated sewer defects detection has become an important trend for better management and maintenance of urban sewer systems. Deep learning technology has developed rapidly and offers an innovative solution for automated detection in engineering applications. However, insufficient data and unbalanced samples have proposed a big challenge to deep learning model training. This study adopts the state-of-the-art Style-based Generative Adversarial Networks (StyleGANs) model and compares the performances of its two variants in producing high-quality synthetic sewer defects images. Seven wellknown CNN models are further fine-tuned and trained using the synthetic images for automated sewer defects detection to examine the effects of StyleGANs on augmenting the detection performance. Results show that both StyleGANs are efficient in producing high-quality images with various styles and high-level details for multiple types of sewer defects. Specifically, the StyleGAN2-Adaptive Discriminator Augmentation (StyleGAN2-ADA) with the aid of Freeze Discriminator (Freeze-D) yields the best model performance. Among the adopted CNN classifiers, Inception_v3 achieves the highest detection accuracy. The mean detection accuracy is 94% (with a specific accuracy of 99.7%, 97%, 95.3% and 84% for tree root, residential wall, disjoint and obstacle, respectively) and confirms the reliability of the StyleGANs' performance. The study shows that StyleGANs provide a promising method to alleviate the limited and uneven dataset problem and can improve the deep learning model performance.

show abstract

Section: Improved Strategymentioning

confidence: 99%

Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier

et al. 2021

View full text Add to dashboard Cite

show abstract

“…GANs are capable of image generation in two categories: low-resolution [4,8,12,13,15,16,23] and high-resolution [14,17,[31][32][33][34][35][36][37][38][39][40][41]. A summary of these approaches is presented in Table 1.…”

Section: Related Workmentioning

confidence: 99%

“…Transfer learning on generative models for limited data has been the subject of study for the last three years [33,34,[38][39][40][41], focusing on evaluating the impact of freezing the lower generator layers [33,34], the lower discriminator layers [39], and both the generator and discriminator lower layers [40], using mainly general purposes datasets of indoors (e.g., LSUN, Bedroons) and faces (e.g., CelebHQ, FFHQ, CelebA). The results show a reduction in the overfitting derived from the knowledge transfer and training time.…”

Section: Related Workmentioning

confidence: 99%

StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

2021

View full text Add to dashboard Cite

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.

show abstract

“…In the case of mode collapse, the GAN generator produces the same image regardless of the input noise, and is not able to recover from this. Several approaches have been developed in recent years to improve the training of a GAN when only few data are available [14][15][16][17][18][19]. In [19], the so-called stochastic discriminator augmentation is used to artificially increase the amount of training data and prevent the generator from learning these augmentations.…”

Section: Introductionmentioning

confidence: 99%

“…However, thousands of images are still required to train a high-resolution GAN like StyleGAN2. In addition to data augmentation, the current research focuses on transfer-learning methods for GANs in cases with few data samples [14][15][16][17]. During the transfer-learning in [16,17] the first layers of the discriminator, the discriminator and generator, are kept fixed.…”

Section: Introductionmentioning

confidence: 99%

Generating Synthetic Sidescan Sonar Snippets Using Transfer-Learning in Generative Adversarial Networks

Steiniger

Kraus

Meisen

2021

JMSE

View full text Add to dashboard Cite

The training of a deep learning model requires a large amount of data. In case of sidescan sonar images, the number of snippets from objects of interest is limited. Generative adversarial networks (GAN) have shown to be able to generate photo-realistic images. Hence, we use a GAN to augment a baseline sidescan image dataset with synthetic snippets. Although the training of a GAN with few data samples is likely to cause mode collapse, a combination of pre-training using simple simulated images and fine-tuning with real data reduces this problem. However, for sonar data, we show that this approach of transfer-learning a GAN is sensitive to the pre-training step, meaning that the vanishing of the gradients of the GAN’s discriminator becomes a critical problem. Here, we demonstrate how to overcome this problem, and thus how to apply transfer-learning to GANs for generating synthetic sidescan snippets in a more robust way. Additionally, in order to further investigate the GAN’s ability to augment a sidescan image dataset, the generated images are analyzed in the image and the frequency domain. The work helps other researchers in the field of sonar image processing to augment their dataset with additional synthetic samples.

show abstract

MineGAN: Effective Knowledge Transfer From GANs to Target Domains With Few Images

Cited by 145 publications

References 13 publications

Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier

Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier

StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Generating Synthetic Sidescan Sonar Snippets Using Transfer-Learning in Generative Adversarial Networks

Contact Info

Product

Resources

About