WarpedGANSpace: Finding non-linear RBF paths in GAN latent space

Abstract

This work addresses the problem of discovering, in an unsupervised manner, interpretable paths in the latent space of pretrained GANs, so as to provide an intuitive and easy way of controlling the underlying generative factors.In doing so, it addresses some of the limitations of the stateof-the-art works, namely, a) that they discover directions that are independent of the latent code, i.e., paths that are linear, and b) that their evaluation relies either on visual inspection or on laborious human labeling. More specifically, we propose to learn non-linear warpings on the latent space, each one parametrized by a set of RBF-based latent space warping functions, and where each warping gives rise to a family of non-linear paths via the gradient of the function. Building on the work of [34], that discovers linear paths, we optimize the trainable parameters of the set of RBFs, so as that images that are generated by codes along different paths, are easily distinguishable by a discriminator network. This leads to easily distinguishable image transformations, such as pose and facial expressions in facial images. We show that linear paths can be derived as a special case of our method, and show experimentally that non-linear paths in the latent space lead to steeper, more disentangled and interpretable changes in the image space than in state-of-the art methods, both qualitatively and quantitatively. We make the code and the pretrained models publicly available at: https://github.com/ chi0tzp/WarpedGANSpace.

Finding Directions in GAN's Latent Space for Neural Face Reenactment

“…Latent GAN space: There is a plethora of works that investigate the existence of interpretable directions in the GAN's latent space [13,23,26,34,35,38,39,46]. Recently, Voynov and Babenko [35], introduce an unsupervised method that is able to discover disentangled linear directions in the latent GAN space by jointly learning the directions and a classifier which learns to predict which direction is responsible for the image transformation.…”

Discovering Multiple and Diverse Directions for Cognitive Image Properties

“…[37] uses local transformation to find a direction conditioned by the latent vector. [30] uses a warping network to learn nonlinear paths in the latent space. [14] proposes a framework that works for domains with more complex nontextured factors of variation using a trainable Neural ODE to find nonlinear directions.…”

Self-Conditioned Generative Adversarial Networks for Image Editing

“…Typically these methods train a network to perform local manipulations on a given latent code [2,20,43]. Others suggest to model the warped manifold of the GAN [38] or traverse this manifold by finding a new local-basis at every step [9]. While such methods have enjoyed relative success, we argue that they aim at solving the problem by attacking the symptom rather than its cause.…”