Transferring GANs: Generating Images from Limited Data

Abstract

Transferring knowledge of pre-trained networks to new domains by means of fine-tuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been studied within the context of generative deep networks. Therefore, we study domain adaptation applied to image generation with generative adversarial networks. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Our results show that using knowledge from pre-trained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when target data is limited. We show that these conclusions can also be drawn for conditional GANs even when the pre-trained model was trained without conditioning. Our results also suggest that density is more important than diversity and a dataset with one or few densely sampled classes is a better source model than more diverse datasets such as ImageNet or Places.

Image Generation From Small Datasets via Batch Statistics Adaptation

“…We used the SNGAN 1 www.gwern.net/Danbooru2018 model used in [21]. Transfer GAN [35]: The pre-trained generator and discriminator are fine-tuned on a small dataset. Transfer GAN (scale and shift): This method is similar to our method, but does not apply supervised training, and instead uses unsupervised training with the discriminator.…”

“…In this subsection, we conducted an experiment to investigate when the generator can be transferred to the target domain. As discussed in 3.2, the diversity of the filter acquired in the pre-trained generator is thought to affect the Anime face Flower Figure 9: Comparison of FID on the anime face dataset (left) and KMMD on the flower dataset (right) between our method, Transfer GAN [35], and "Update all". Note that it is meaningless to compare the performance between different dataset sizes because the data distributions for each dataset size are different.…”

“…In this section, we show the generative results from Transfer GAN [35] and "Update all" for each data size, which transfers prior knowledge of generative models similarly to our method. We used human face, anime face, and flower images for training and chose 25, 50, 100, and 500 as the data size.…”

SAR: learning cross-language API mappings with little knowledge

“…Recent progresses in domain adaption alleviate the need of parallel corpora [6,9,10]. In an application to image learning, domain adaptation through GAN has shown benefit to transfer the models from other dataset as pre-training models when training on smaller dataset [32], which provides the technical foundation for our work.…”

Improving Deep Interactive Evolution with a Style-Based Generator for Artistic Expression and Creative Exploration

“…This paper explores the intersection of generative adversarial networks (GANs) and interactive evolutionary computation (IEC) within the context of artistic artifact generation and creative expression. GANs are state-of-the-art generative models that have been applied in artifact generation involving video, audio, 3D models, virtual ambiance and videogames [ 11 , 12 , 13 , 14 ], but have found most of their success in the generation of 2D images [ 15 ]. GANs generate new artifacts by sampling from a learned latent space; however, this sampling process is mostly done stochastically and offers little control over the final output.…”