Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing

Stitch it in Time: GAN-Based Facial Editing of Real Videos

“…Multiple works have studied inversion in the context of StyleGAN. They either directly optimize the latent vector to reproduce a specific image [1,2,6,15,33,46,56], or train an efficient encoder over large collection of images [4,5,16,19,23,29,32,40,53]. Typically, direct optimization is more accurate, but encoders are faster at inference.…”

FEAT: Face Editing with Attention

“…Recently, domain adaptation methods [18,38,56] that build upon StyleGAN have also been proposed to achieve style transfer. Several works [11,19,25] have focused on local semantically-aware edits to a target output image by encoding the local semantics of images into the StyleGAN latent space. Additionally, in order to support real image manipulations, GAN inversion [1,4,5,33,34,39,43,54,57] has been adopted to inversely project a given image into the latent space of a pretrained GAN generator, and then the obtained codes can be edited in a semantically meaningful manner.…”

Region-Based Semantic Factorization in GANs

“…Among all the applications of GANs, image local editing earns a number of audiences considering its interactivity and practical usage. One straightforward way of controlling the synthesis of a certain image region is to make the GAN generator spatially aware during training [26,28]. An alternative way is to first segment the synthesis results and then manipulate (e.g., swap) the intermediate feature maps at the region of interest [4,10,38,47].…”

“…However, all these approaches tend to perform editing only from the instance level instead of the semantic level. Taking face local manipulation as an example, these methods are capable of harmonizing the eyes of one person to another [10,26,28,38] yet fail to make a person close the eyes. Meanwhile, they require users to specify spatial masks for each editing (e.g., the eyes of a person may not always locate at the same spatial position in different images), making them hard to generalize to all samples.…”