PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

Jiang, Wentao; Liu, Si; Gao, Chen; Cao, Jie; He, Ran; Feng, Jiashi; Yan, Shuicheng

doi:10.1109/cvpr42600.2020.00524

Cited by 129 publications

(107 citation statements)

References 17 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…We investigated the trend of visual changes with each layer in our material (Fig 6) in which we used StyleGAN2. It shows the clear trends that the Coarse layers (1-4) correspond to the shape and posture, the Middle layers (5-8) correspond to physiognomy, and the Fine layers (9)(10)(11)(12)(13)(14)(15)(16)(17)(18) correspond to the color and texture of the face image. The fact that the Coarse and Middle layers have latent image features that bring about greater visual changes is consistent with the result that the RMSE in CP was lower in those layers.…”

Section: Discussionmentioning

confidence: 99%

Human Latent Metrics: Perceptual and Cognitive Response Corresponds to Distance in GAN Latent Space

Kasahara

Ienaga

Shimizu

et al. 2022

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) generate high-dimensional vector spaces (latent spaces) that can interchangeably represent vectors as images. Advancements have extended their ability to computationally generate images indistinguishable from real images such as faces, and more importantly, manipulate images using their inherit vector values in the latent space. This interchangeability of latent vector has the potential to calculate not only distance in the latent space, but also human perceptual and cognitive distance toward images, i.e., how humans perceive and recognize images. However, it is still unclear how the distance in the latent space corresponds to human perception and cognition. Our studies investigated the correspondence between latent vectors and human perception or cognition through psycho-visual experiments that manipulates the latent vectors of face images. In the perception study, a change perception (CP) task was utilized to examine whether participants could perceive visual changes in face images before and after moving an arbitrary distance in the latent space. In the cognition study, a face cognition (FC) task was utilized to examine whether the participants could recognize a face as the same, even after moving an arbitrary distance in the latent space. The results showed that CP and cognition for face images clearly correlates to the distance in the latent space, which can be modeled with a logistic function. We also investigated how the internal layered structure of the latent space correlates to human response by calculating the regression residual error in each layer. As a result, we observed different residual error trends pertaining to CP and FC. Our experiments show that the distance between face images in the latent space corresponds to human perception and cognition for visual changes in face imagery, and additionally indicates that perception and cognition correspond with the latent space differently. By utilizing our methodology, it will be possible to interchangeably convert between the distance in the latent space and the metric of human perception and cognition, potentially leading to image processing that better reflects human perception and cognition.

show abstract

Section: Discussionmentioning

confidence: 99%

Human Latent Metrics: Perceptual and Cognitive Response Corresponds to Distance in GAN Latent Space

Kasahara

Ienaga

Shimizu

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…An early paper in computer graphics showed how conditioning on sketches can produce good results for terrain modeling [Guérin et al 2017]. Specialized image manipulation techniques, such as makeup transfer PSGAN [Jiang et al 2019] or hair editing [Tan et al 2020] are also very useful, as faces are an important class of images. A very challenging style transfer technique is the transformation of input photographs to obtain caricatures ].…”

Section: Related Workmentioning

confidence: 99%

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows

et al. 2021

View full text Add to dashboard Cite

High-quality, diverse, and photorealistic images can now be generated by unconditional GANs (e.g., StyleGAN). However, limited options exist to control the generation process using (semantic) attributes while still preserving the quality of the output. Further, due to the entangled nature of the GAN latent space, performing edits along one attribute can easily result in unwanted changes along other attributes. In this article, in the context of conditional exploration of entangled latent spaces, we investigate the two sub-problems of attribute-conditioned sampling and attribute-controlled editing. We present StyleFlow as a simple, effective, and robust solution to both the sub-problems by formulating conditional exploration as an instance of conditional continuous normalizing flows in the GAN latent space conditioned by attribute features. We evaluate our method using the face and the car latent space of StyleGAN, and demonstrate fine-grained disentangled edits along various attributes on both real photographs and StyleGAN generated images. For example, for faces, we vary camera pose, illumination variation, expression, facial hair, gender, and age. Finally, via extensive qualitative and quantitative comparisons, we demonstrate the superiority of StyleFlow over prior and several concurrent works. Project Page and Video: https://rameenabdal.github.io/StyleFlow .

show abstract

“… Gaming: It can also be used in Mixed Reality (MR) games wherein the real world seen from the MR headset will change based on the style used for the game [40], [41].  Fashion industry: NST can find applications in the fashion industry where designers and consumers can use it to overlay items while designing or trying them [42], [43].…”

Section: Possible Future Applications Of Nstmentioning

confidence: 99%

Neural Style Transfer: A Critical Review

et al. 2021

View full text Add to dashboard Cite

Neural Style Transfer (NST) is a class of software algorithms that allows us to transform scenes, change/edit the environment of a media with the help of a Neural Network. NST finds use in image and video editing software allowing image stylization based on a general model, unlike traditional methods. This made NST a trending topic in the entertainment industry as professional editors/media producers create media faster and offer the general public recreational use. In this paper, the current progress in Neural Style Transfer with all related aspects such as still images and videos is presented critically. The authors looked at the different architectures used and compared their advantages and limitations. Multiple literature reviews focus on the Neural Style Transfer of images and cover Generative Adversarial Networks (GANs) that generate video. As per the authors' knowledge, this is the only research article that looks at image and video style transfer, particularly mobile devices with high potential usage. This article also reviewed the challenges faced in applying for video neural style transfer in realtime on mobile devices and presents research gaps with future research directions. NST, a fascinating deep learning application, has considerable research and application potential in the coming years.

show abstract

PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

Cited by 129 publications

References 17 publications

Human Latent Metrics: Perceptual and Cognitive Response Corresponds to Distance in GAN Latent Space

Human Latent Metrics: Perceptual and Cognitive Response Corresponds to Distance in GAN Latent Space

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows

Neural Style Transfer: A Critical Review

Contact Info

Product

Resources

About