Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning

Deng, Yu; Yang, Jiaolong; Chen, Dong; Wen, Fang; Tong, Xin

doi:10.1109/cvpr42600.2020.00520

Cited by 240 publications

(173 citation statements)

References 28 publications

Supporting

Mentioning

172

Contrasting

Order By: Relevance

“…It is shown that the feature maps at different resolutions of the StyleGAN architecture characterize the image styles at different scales and control the generation in a coarse-to-fine fashion. With StyleGAN, image manipulation can be achieved by embedding a real photo into the StyleGAN latent space and editing the embedded latent code for re-generation [Abdal et al 2019[Abdal et al , 2020Deng et al 2020;Karras et al 2020b;Tewari et al 2020;Zhu et al 2020]. Semantically-meaningful editing in GAN latent space has also been studied [Bau et al 2019;Brock et al 2016;Creswell and Bharath 2018;Richardson et al 2020;Yeh et al 2017;Zhu et al 2016].…”

Section: Image Generation and Editing Using Styleganmentioning

confidence: 99%

StyleCariGAN

Jang

Jung

et al. 2021

ACM Trans. Graph.

Self Cite

View full text Add to dashboard Cite

Korea(a) Photo (b) CariGANs (c) WarpGAN (d) AutoToon (e) Ours (f) Ours + InterFaceGANFig. 1. Comparison of previous caricature generation methods with ours. Different state-of-the-art caricature generation methods are compared to our proposed StyleCariGAN; CariGANs [Cao et al. 2018] (b); WarpGAN [Shi et al. 2019] (c); AutoToon [Gong et al. 2020] (d). Our method (e) can be used with other StyleGAN-based image editing methods, e.g., InterFaceGAN [Shen et al. 2020] (f).

show abstract

Section: Image Generation and Editing Using Styleganmentioning

confidence: 99%

StyleCariGAN

Jang

Jung

et al. 2021

ACM Trans. Graph.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Face image synthesis has achieved tremendous success recently with Generative Adversarial Networks (GANs) [Goodfellow et al, 2014]. State-of-the-art GAN models [Karras et al, 2019;Karras et al, 2020] can generate high-fidelity face images from the learned latent space.…”

Section: Oursmentioning

confidence: 99%

“…A successful editing should not only output high-quality results with accurate target attribute, but also well preserve all other image content characterized by the complementary attributes. Face attribute editing has attracted much attention in recent years and numerous algorithms have been proposed [Shen and Liu, 2017;Choi et al, 2018;Bahng et al, 2020;Awiszus et al, 2019;Gu et al, 2019]. Notwithstanding the…”

Section: Introductionmentioning

confidence: 99%

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search

Han

Yang

2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Recent works have shown that a rich set of semantic directions exist in the latent space of Generative Adversarial Networks (GANs), which enables various facial attribute editing applications. However, existing methods may suffer poor attribute variation disentanglement, leading to unwanted change of other attributes when altering the desired one. The semantic directions used by existing methods are at attribute level, which are difficult to model complex attribute correlations, especially in the presence of attribute distribution bias in GAN's training set. In this paper, we propose a novel framework (IALS) that performs Instance-Aware Latent-Space Search to find semantic directions for disentangled attribute editing. The instance information is injected by leveraging the supervision from a set of attribute classifiers evaluated on the input images. We further propose a Disentanglement-Transformation (DT) metric to quantify the attribute transformation and disentanglement efficacy and find the optimal control factor between attribute-level and instance-specific directions based on it. Experimental results on both GAN-generated and real-world images collectively show that our method outperforms state-of-the-art methods proposed recently by a wide margin. Code is available at https://github.com/yxuhan/IALS.

show abstract

“…Bepler et al [2] and Detlefsen and Hauberg [3] performed disentanglement on image data by separating the latent variables into two groups assigned to appearance and perspective. Deng et al [10] also developed a disentanglement method for facial images by assigning parameters of a three-dimensional morphable face model [41] to the latent variables.…”

Section: Disentangled Representation Learningmentioning

confidence: 99%

“…In many DRL methods used in previous studies, attempts were made to discover factors of variation without any prior information or supervision. Although remarkable results were obtained by these unsupervised DRL methods using toy datasets such as dSprites [22] and 3D Shapes [23], there is no guarantee that each latent variable corresponds to a single semantically meaningful factor of variation without any inductive bias [10], [24], [25]. Hence, recent DRL studies have focused on introduction to a model of an explicit prior that imposes constraints or regulariza-tions based on the underlying structure of complicated realworld images [26], [27], such as translation and rotation [2], [28], hierarchical features [8], [9], [29] and domain-specific knowledge [10].…”

Section: Introductionmentioning

confidence: 99%

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior

et al. 2021

View full text Add to dashboard Cite

We propose a novel method that can learn easy-to-interpret latent representations in realworld image datasets using a VAE-based model by splitting an image into several disjoint regions. Our method performs object-wise disentanglement by exploiting image segmentation and alpha compositing. With remarkable results obtained by unsupervised disentanglement methods for toy datasets, recent studies have tackled challenging disentanglement for real-world image datasets. However, these methods involve deviations from the standard VAE architecture, which has favorable disentanglement properties. Thus, for disentanglement in images of real-world image datasets with preservation of the VAE backbone, we designed an encoder and a decoder that embed an image into disjoint sets of latent variables corresponding to objects. The encoder includes a pre-trained image segmentation network, which allows our model to focus only on representation learning while adopting image segmentation as an inductive bias. Evaluations using real-world image datasets, CelebA and Stanford Cars, showed that our method achieves improved disentanglement and transferability.INDEX TERMS Alpha blend, disentanglement, image segmentation, real-world image, representation learning.

show abstract

Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning

Cited by 240 publications

References 28 publications

StyleCariGAN

StyleCariGAN

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior

Contact Info

Product

Resources

About