Facial Attribute Transformers for Precise and Robust Makeup Transfer

Wan, Zhaoyi; Chen, Haoran; Zhang, Jielei; Jiang, Wentao; Yao, Cong; Luo, Jiebo

doi:10.1109/wacv51458.2022.00317

Cited by 17 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, there have been many approaches based on generative adversarial networks. [1][2][3][4][5] BeautyGAN 1 was one of the first methods to use GAN for makeup transfer, which uses a pixel-level histogram loss to achieve instance-level makeup transfer. PSGAN 2 utilizes the face analysis map and facial coordinate points to construct the pixel-level correspondence between the source image and the reference image, so as to solve the problem of face misalignment between different head poses and facial expressions.…”

Section: Makeup Transfermentioning

confidence: 99%

“…In recent years, there have been many approaches based on generative adversarial networks 1‐5 . BeautyGAN 1 was one of the first methods to use GAN for makeup transfer, which uses a pixel‐level histogram loss to achieve instance‐level makeup transfer.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, generative adversarial network (GAN) technology has been widely applied to various computer vision tasks due to its ability to produce high-quality realistic images. GAN-based facial makeup transfer methods [1][2][3][4][5] can significantly improve the makeup transfer effect. On the other hand, there are many kinds of makeup styles, including daily simple makeup, stage performance makeup and personalized extreme makeup.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Personalized facial makeup transfer based on outline correspondence

Wang

2023

Computer Animation & Virtual

View full text Add to dashboard Cite

Most existing makeup transfer techniques focus on light makeup styles, which limits the task of makeup transfer to color manipulation issues such as eye shadow and lip gloss. However, the makeup in real life is diverse and personalized, not only the most basic foundation, eye makeup, but also the painted patterns on the face, jewelry decoration and other personalized makeup. Inspired by the painting steps of drawing the outline first and then coloring, we propose a makeup transfer network for personalized makeup, which realizes face makeup transfer by learning outline correspondence. Specifically, we propose the outline feature extraction module and outline loss that can promote outline correspondence. Our network can not only transfer daily light makeup, but also complete transfer for complex facial painting patterns. Experiments show that our method can obtain visually more accurate makeup transfer results. Quantitative and qualitative experimental results show that the method proposed in this paper achieves superior results in extreme makeup transfer compared to the state‐of‐the‐art methods.

show abstract

Section: Makeup Transfermentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Personalized facial makeup transfer based on outline correspondence

Wang

2023

Computer Animation & Virtual

View full text Add to dashboard Cite

show abstract

“…Recently, MPViT [66] explores multi-scale patch embedding and multi-path structure that enable both fine and coarse feature representations simultaneously. Benefiting from advances in basic vision transformer models, many task-specific models are proposed and achieve significant progress in downstream vision tasks, e.g., object detection [13,134,43,22], semantic segmentation [130,17,98,116,126,29,28], generative adversarial network [59,102,58], low-level vision [16,72,127], video understanding [81,7,94,104,118], self-supervised learning [2,24,80,53,4,14,38,23,117,110,3,45], neural architecture search [103,67,15,19,20], etc. Inspired by practical improvements in EA variants, this work migrates them to Transformer improvement and designs a powerful visual model with higher precision and efficiency than contemporary works.…”

Section: Vision Transformersmentioning

confidence: 99%

EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm

Zhang¹,

Li²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Motivated by biological evolution, this paper explains the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derives that both have consistent mathematical formulation. Then inspired by effective EA variants, we propose a novel pyramid EATFormer backbone that only contains the proposed EA-based Transformer (EAT) block, which consists of three residual parts, i.e., Multi-Scale Region Aggregation (MSRA), Global and Local Interaction (GLI), and Feed-Forward Network (FFN) modules, to model multi-scale, interactive, and individual information separately. Moreover, we design a Task-Related Head (TRH) docked with transformer backbone to complete final information fusion more flexibly and improve a Modulated Deformable MSA (MD-MSA) to dynamically model irregular locations. Massive quantitative and quantitative experiments on image classification, downstream tasks, and explanatory experiments demonstrate the effectiveness and superiority of our approach over State-Of-The-Art (SOTA) methods. E.g., our Mobile (1.8M), Tiny (6.1M), Small (24.3M), and Base (49.0M) models achieve 69.4, 78.4, 83.1, and 83.9

show abstract

“…Therefore, research of FBP is scientifically important for understanding the perception mechanism of human brain and simulating human intelligence. Simultaneously, exploring how to better interpret, quantify and predict beauty will help people understand and describe beauty more scientifically and objectively, further promoting the rapid development of related industries, such as makeup evaluation (Wei et al 2022), makeup transfer (Wan et al 2022), personalization recommendation (Lin et al 2019a, b) and cosmetic surgery planning (Xie et al 2015). In recent years, scholars have been working hard to explore deep learning and use it for FBP.…”

Section: Introductionmentioning

confidence: 99%

Facial beauty prediction fusing transfer learning and broad learning system

et al. 2022

View full text Add to dashboard Cite

Facial beauty prediction (FBP) is an important and challenging problem in the fields of computer vision and machine learning. Not only it is easily prone to overfitting due to the lack of large-scale and effective data, but also difficult to quickly build robust and effective facial beauty evaluation models because of the variability of facial appearance and the complexity of human perception. Transfer Learning can be able to reduce the dependence on large amounts of data as well as avoid overfitting problems. Broad learning system (BLS) can be capable of quickly completing models building and training. For this purpose, Transfer Learning was fused with BLS for FBP in this paper. Firstly, a feature extractor is constructed by way of CNNs models based on transfer learning for facial feature extraction, in which EfficientNets are used in this paper, and the fused features of facial beauty extracted are transferred to BLS for FBP, called E-BLS. Secondly, on the basis of E-BLS, a connection layer is designed to connect the feature extractor and BLS, called ER-BLS. Finally, experimental results show that, compared with the previous BLS and CNNs methods existed, the accuracy of FBP was improved by E-BLS and ER-BLS, demonstrating the effectiveness and superiority of the method presented, which can also be widely used in pattern recognition, object detection and image classification.

show abstract

Facial Attribute Transformers for Precise and Robust Makeup Transfer

Cited by 17 publications

References 18 publications

Personalized facial makeup transfer based on outline correspondence

Personalized facial makeup transfer based on outline correspondence

EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm

Facial beauty prediction fusing transfer learning and broad learning system

Contact Info

Product

Resources

About