Transforming and Projecting Images into Class-Conditional Generative Networks

FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization

“…The optimization is solved with Adam [17] in [20,39]. We truncate z to [−2, 2] as a typical practice when using BigGAN [4,16].…”

“…In this work, we adopt the widely-used Adam [17] optimizer. Some recent works recommended to use gradient-free optimizers, like BasinCMA [3,16,32], for optimizing in GAN spaces due to the high non-convexity. However, our study shows that BasinCMA tends to cast a higher computation cost than Adam, because BasinCMA requires a Algorithm 1 FuseDream (with single image generation)…”

“…The space of images of the CLIP+GAN approach is limited by the representation power of the GAN we use. This makes the method difficult to generate out-of-distribution images and prune to inherit the data biases e.g., center, spatial and color bias [2,16], from the original training set of the GAN. We propose composed generation, which expands the image space and reduce the data bias by composing two images generated by GAN to gain highly flexibility.…”

“…In practice, we combine (6) with Adam by treating v t as the gradient direction. In addition, we obtain the final composed image by applying Poisson blending on I f g and I bg to yield even smoother image I following [16]. Our algorithm is summarized in Alg.…”

Fast Inverse Mapping of Face GANs

“…Recovered latent vectors can be used as a measure of evaluating GAN performance [2] as well as a method to find out about the features a GAN has learned from its training dataset. Moreover, it is possible to modify images towards a desired direction [3] [4] using recovered latent vectors, for example applying styles to human faces. In addition, linear operations on latent vectors result in meaningful changes in generated images [5].…”