Self-Distilled StyleGAN: Towards Generation from Internet Photos

Abstract

StyleGAN is known to produce high-fidelity images, while also offering unprecedented semantic editing. However, these fascinating abilities have been demonstrated only on a limited set of datasets, which are usually structurally aligned and well curated. In this paper, we show how StyleGAN can be adapted to work on raw uncurated images collected from the Internet. Such image collections impose two main challenges to StyleGAN: they contain many outlier images, and are characterized by a multi-modal distribution. Training StyleGAN on such raw image collections results in degraded image synthesis quality. To meet these challenges, we proposed a StyleGAN-based self-distillation approach, which consists of two main components: (i) A generative-based self-filtering of the dataset to eliminate outlier images, in order to generate an adequate training set, and (ii) Perceptual clustering of the generated images to detect the inherent data modalities, which are then employed to improve StyleGAN's "truncation trick" in the image synthesis process. The presented technique enables the generation of high-quality images, while minimizing the loss in diversity of the data. Through qualitative and quantitative evaluation, we demonstrate the power of our approach to new challenging and diverse domains collected from the Internet. New datasets and pre-trained models are available in our project website 1 .

ArtFID: Quantitative Evaluation of Neural Style Transfer

“…As an example, consider the closely-related field of image synthesis, where quantitative measurements for image quality are used, not just for evaluation, but also as a core part of the approach. Recent examples include work by Mokady et al [56], where Fréchet Inception Distance (FID) [26] and Learned Perceptual Image Patch Similarity (LPIPS) [86] measurements are employed for self-filtering a collection of images, work by Karras et al [34] that use the FID to detect GAN [24] overfitting, or a large-scale GAN study by Lucic et al [53] that utilize the FID to compare the sensitivity of different GAN models to hyper-parameters. A quantitative measurement of stylization performance would facilitate similar studies to further analyze and improve style transfer methods.…”