Comparing the latent space of generative models

Abstract

Different encodings of datapoints in the latent space of latent-vector generative models may result in more or less effective and disentangled characterizations of the different explanatory factors of variation behind the data. Many works have been recently devoted to the exploration of the latent space of specific models, mostly focused on the study of how features are disentangled and of how trajectories producing desired alterations of data in the visible space can be found. In this work we address the more general problem of comparing the latent spaces of different models, looking for transformations between them. We confined the investigation to the familiar and largely investigated case of generative models for the data manifold of human faces. The surprising, preliminary result reported in this article is that (provided models have not been taught or explicitly conceived to act differently) a simple linear mapping is enough to pass from a latent space to another while preserving most of the information. This is full of consequences for representation learning, potentially paving the way to the transformation of editing trajectories from one space to another, or the adaptation of disentanglement techniques between different generative domains.

ELODIN: Naming Concepts in Embedding Spaces

“…Many works have used the latent spaces of GANs (especially StyleGAN [17]) to invert and edit images [30]. More recently, there has been some investigation in Diffusion models' latent spaces as well [3,4].…”

Portrait Reification with Generative Diffusion Models

“…This is particularly effective for reverse diffusion techniques, since they have a larger sample diversity and introduce much less artifacts in the generative process than different generative techniques [6]. GANs suffer from the well-known mode collapse phenomenon [7], privileging realism over diversity, and essentially preventing the encoding of arbitrary samples from the true distribution [8]. VAEs offer a more adequate coverage of the data distribution but, in comparison with alternative generative techniques, they usually produce images with a characteristic and annoying blurriness very hard to correct [9,10].…”