Image embedding for denoising generative models

Abstract

Denoising Diffusion models are gaining increasing popularity in the field of generative modeling for several reasons, including the simple and stable training, the excellent generative quality, and the solid probabilistic foundation. In this article, we address the problem of embedding an image into the latent space of Denoising Diffusion Models, that is finding a suitable “noisy” image whose denoising results in the original image. We particularly focus on Denoising Diffusion Implicit Models due to the deterministic nature of their reverse diffusion process. As a side result of our investigation, we gain a deeper insight into the structure of the latent space of diffusion models, opening interesting perspectives on its exploration, the definition of semantic trajectories, and the manipulation/conditioning of encodings for editing purposes. A particularly interesting property highlighted by our research, which is also characteristic of this class of generative models, is the independence of the latent representation from the networks implementing the reverse diffusion process. In other words, a common seed passed to different networks (each trained on the same dataset), eventually results in identical images.

A Generative Approach to Person Reidentification

“…The feasibility of inverting diffusion models through appropriately trained neural networks has been explored in many of our prior works. The overall idea was presented in [ 57 ]; this approach was applied in [ 58 ] for the “reification” of artistic portraits by embedding a portrait into a latent space of human faces and reconstructing the closest real approximation. In [ 59 ], the diffusion inversion was utilized to create a trajectory in the latent space that induces a smooth rotation effect on human faces.…”

“…A key feature of these models is their fully deterministic reverse diffusion process. This attribute is crucial for applications that require embedding the output back into its latent representation, as discussed in [ 57 ]. Another significant advantage of Implicit Diffusion Models is their efficiency, typically requiring only a minimal number of iterations (about 10, as reported in [ 58 , 59 ])—a stark contrast to other techniques that may need thousands of iterations.…”

“…Regarding the inversion networks, Img2Noise entails an image-to-image process, as the noise maintains the same spatial dimension as the input image. Based on prior studies [ 57 ], the U-Net architecture has demonstrated superior performance for such tasks.…”

“…Another significant advantage of Implicit Diffusion Models is their efficiency, typically requiring only a minimal number of iterations (about 10, as reported in [58,59])-a stark contrast to other techniques that may need thousands of iterations. Additionally, their remarkable conditioning capabilities have been demonstrated in the field of precipitation forecasting [56], where the goal is to predict the probability distribution of a specific atmospheric parameter based on recent weather conditions.…”