CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions

TextIR: A Simple Framework for Text-based Editable Image Restoration

“…TediGAN [51] proposes to generate an image corresponding to a given text by training an encoder to map the text into the StyleGAN latent space. Several recent works [1,10] jointly utilize pre-trained generative models [4, 12,14] and CLIP to steer the generated result towards the desired target description. More recently, diffusion models (DM) [17,33,41] achieves state-of-the-art results on test-to-image synthesis by decomposing the image formation process into a sequential application of denoising autoencoders.…”

TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision

“…With the introduction of StyleGAN [14][15][16] mapping networks, the input random noise can be first mapped to another latent space that has disentangled semantics, then the model can generate images with better quality. Further, exploring the latent space of StyleGAN has been proved useful by several works [1,17,33] in text-driven image synthe- sis and manipulation tasks, where they utilize the pretrained vision-language model CLIP [37] model to manipulate pretrained unconditional StyleGAN networks. In order to relieve the need for paired text data during the training phase, Lafite [55] proposes to adopt the image CLIP embeddings as training input while using text CLIP embedding during the inference phase.…”

CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not

“…Such common extensions include retrieval [4], image generation [11], continual learning [61], object detection, few-shot recognition [22], semantic segmentation [47], etc. Others leveraged CLIP's image/text encoders with StyleGAN [45] to enable intuitive text-based semantic image manipulation [1]. In our work we adapt CLIP for zero-shot FG-SBIR in a cross-category setting.…”