BeautyGAN

Li, Tingting; Qian, Ruihe; Dong, Chao; Liu, Si; Yan, Qing; Zhu, Wenwu; Li, Lin

doi:10.1145/3240508.3240618

Cited by 165 publications

(19 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Style transfer [4] is an image-to-image translation technique that aims to separate and recombine the content and style information of images. Built on the style transfer framework, makeup transfer [1,5,12,19,23] is proposed to transfer the makeup style of the reference image to the source image while keeping the result of face recognition unchanged. Both style transfer and makeup transfer rely on the cycle consistency loss or its variants [43,45] to maintain the stability of source images.…”

Section: Style Transfer and Makeup Transfermentioning

confidence: 99%

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

Liu

Zhang

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

Section: Style Transfer and Makeup Transfermentioning

confidence: 99%

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

Liu

Zhang

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Then, this product representation is decoded and rendered on the source image using generative models such as generative advesarial networks (GANs) [GPAM ∗ 20] or variational autoencoders (VAEs) [KW14]. In particular, this idea has been successfully used for makeup transfer [LQD ∗ 18, JLG ∗ 20], hair synthesis [SDS ∗ 21, KCP ∗ 21] and is rapidly emerging in the field of fashion articles [JB17]. Recent methods attempt to provide controllable rendering [KPGB20], or propose to leverage additional scene information in their models, such as segmentation masks for fashion items [CPLC21,GSG ∗ 21] or UV maps for makeup [NTH21].…”

Section: Related Workmentioning

confidence: 99%

“…This task consists in extracting a product appearance from a single reference image and synthesizing it on the image of another person. However, existing methods in this domain are often based on large generative networks that suffer from temporal inconsistencies, and cannot be used to process a video stream in real‐time on mobile devices [KJB ∗ 21,LQD ∗ 18].…”

Section: Introductionmentioning

confidence: 99%

Real‐time Virtual‐Try‐On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers

Kips

Jiang²,

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Augmented reality applications have rapidly spread across online retail platforms and social media, allowing consumers to virtually try-on a large variety of products, such as makeup, hair dying, or shoes. However, parametrizing a renderer to synthesize realistic images of a given product remains a challenging task that requires expert knowledge. While recent work has introduced neural rendering methods for virtual try-on from example images, current approaches are based on large generative models that cannot be used in real-time on mobile devices. This calls for a hybrid method that combines the advantages of computer graphics and neural rendering approaches. In this paper, we propose a novel framework based on deep learning to build a real-time inverse graphics encoder that learns to map a single example image into the parameter space of a given augmented reality rendering engine. Our method leverages self-supervised learning and does not require labeled training data, which makes it extendable to many virtual try-on applications. Furthermore, most augmented reality renderers are not differentiable in practice due to algorithmic choices or implementation constraints to reach real-time on portable devices. To relax the need for a graphics-based differentiable renderer in inverse graphics problems, we introduce a trainable imitator module. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non-differentiable renderer. We propose a novel rendering sensitivity loss to train the imitator, which ensures that the network learns an accurate and continuous representation for each rendering parameter. Automatically learning a differentiable renderer, as proposed here, could be beneficial for various inverse graphics tasks. Our framework enables novel applications where consumers can virtually try-on a novel unknown product from an inspirational reference image on social media. It can also be used by computer graphics artists to automatically create realistic rendering from a reference product image. CCS Concepts• Computing methodologies → Computer vision; Machine learning; Computer graphics; Recently, a novel family of methods based on neural rendering has introduced the task of image based virtual try-on, which brings new perspectives for this problem [TFT * 20]. This task consists in extracting a product appearance from a single reference image and synthesizing it on the image of another person. However, existing methods in this domain are often based on large genera-

show abstract

“…Human evaluation studies, on the other hand, tend to focus on comparing the outputs of different algorithms, again ignoring the importance of context on the evaluation results. Typically, participants are asked to rank pictures produced using different algorithms from best to worst (Li et al 2018;Liu et al 2017;Zhou et al 2019b) or rate the pictures by user perception metrics, such as realism, overall quality, and identity (Yin et al 2017;Zhou et al 2019b). For example, Li et al (2018) recruited 84 volunteers to rank three generated images out of 10 non-makeup and 20 makeup test images based on quality, realism, and makeup style similarity.…”

Section: Lack Of Evaluation Studies For Artificial Picturesmentioning

confidence: 99%

“…Typically, participants are asked to rank pictures produced using different algorithms from best to worst (Li et al 2018;Liu et al 2017;Zhou et al 2019b) or rate the pictures by user perception metrics, such as realism, overall quality, and identity (Yin et al 2017;Zhou et al 2019b). For example, Li et al (2018) recruited 84 volunteers to rank three generated images out of 10 non-makeup and 20 makeup test images based on quality, realism, and makeup style similarity. Lee et al (2018) employed a similar approach by asking users which image is more realistic out of samples created using different generation methods.…”

Section: Lack Of Evaluation Studies For Artificial Picturesmentioning

confidence: 99%

Using artificially generated pictures in customer-facing systems: an evaluation study with data-driven personas

Salminen

Jung

Kamel

et al. 2020

Behaviour & Information Technology

View full text Add to dashboard Cite

We conduct two studies to evaluate the suitability of artificially generated facial pictures for use in a customer-facing system using data-driven personas. STUDY 1 investigates the quality of a sample of 1,000 artificially generated facial pictures. Obtaining 6,812 crowd judgments, we find that 90% of the images are rated medium quality or better. STUDY 2 examines the application of artificially generated facial pictures in data-driven personas using an experimental setting where the highquality pictures are implemented in persona profiles. Based on 496 participants using 4 persona treatments (2 × 2 research design), findings of Bayesian analysis show that using the artificial pictures in persona profiles did not decrease the scores for Authenticity, Clarity, Empathy, and Willingness to Use of the data-driven personas.

show abstract

BeautyGAN

Cited by 165 publications

References 13 publications

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

Real‐time Virtual‐Try‐On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers

Using artificially generated pictures in customer-facing systems: an evaluation study with data-driven personas

Contact Info

Product

Resources

About