Mask-aware photorealistic facial attribute manipulation

Sun, Ren; Huang, Chen; Shi, Jianping; Ma, Lizhuang

doi:10.1007/s41095-021-0219-7

Cited by 19 publications

(5 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enable control over generated images, conditional GANs (cGANs) integrate category information into the generation process, facilitating control over the category of the generated images. Various methods [5][6][7][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38] employ two-dimensional label maps, such as segmentations or sketch maps, to control the generation and editing of images.…”

Section: D-aware Image Synthesismentioning

confidence: 99%

Multi3D: 3D-aware multimodal image synthesis

Zhou,

Yuan,

2024

Comp. Visual Media

View full text Add to dashboard Cite

Abstract3D-aware image synthesis has attained high quality and robust 3D consistency. Existing 3D controllable generative models are designed to synthesize 3D-aware images through a single modality, such as 2D segmentation or sketches, but lack the ability to finely control generated content, such as texture and age. In pursuit of enhancing user-guided controllability, we propose Multi3D, a 3D-aware controllable image synthesis model that supports multi-modal input. Our model can govern the geometry of the generated image using a 2D label map, such as a segmentation or sketch map, while concurrently regulating the appearance of the generated image through a textual description. To demonstrate the effectiveness of our method, we have conducted experiments on multiple datasets, including CelebAMask-HQ, AFHQ-cat, and shapenet-car. Qualitative and quantitative evaluations show that our method outperforms existing state-of-the-art methods.

show abstract

Section: D-aware Image Synthesismentioning

confidence: 99%

Multi3D: 3D-aware multimodal image synthesis

Zhou,

Yuan,

2024

Comp. Visual Media

View full text Add to dashboard Cite

show abstract

“…Xiao et al [ 98 ] presented a multi-attribute manipulation GANs-based system. Moreover, spatial attention in GANs [ 99 ], variational autoencoder (VAE) + GANs [ 100 ], multi-domain GANs [ 101 ], geometry-aware GANs [ 102 ], mask-guided GANs [ 103 ], 3D face morphable model [ 104 ], and GIMP animation [ 105 ] based methods have been designed.…”

Section: Deepfake Generation and Detectionmentioning

confidence: 99%

Deepfakes Generation and Detection: A Short Survey

Akhtar

2023

J. Imaging

View full text Add to dashboard Cite

Advancements in deep learning techniques and the availability of free, large databases have made it possible, even for non-technical people, to either manipulate or generate realistic facial samples for both benign and malicious purposes. DeepFakes refer to face multimedia content, which has been digitally altered or synthetically created using deep neural networks. The paper first outlines the readily available face editing apps and the vulnerability (or performance degradation) of face recognition systems under various face manipulations. Next, this survey presents an overview of the techniques and works that have been carried out in recent years for deepfake and face manipulations. Especially, four kinds of deepfake or face manipulations are reviewed, i.e., identity swap, face reenactment, attribute manipulation, and entire face synthesis. For each category, deepfake or face manipulation generation methods as well as those manipulation detection methods are detailed. Despite significant progress based on traditional and advanced computer vision, artificial intelligence, and physics, there is still a huge arms race surging up between attackers/offenders/adversaries (i.e., DeepFake generation methods) and defenders (i.e., DeepFake detection methods). Thus, open challenges and potential research directions are also discussed. This paper is expected to aid the readers in comprehending deepfake generation and detection mechanisms, together with open issues and future directions.

show abstract

“…However, MaskGAN just allows global style transfer. Sun et al [40] use partial dilated layers to modify a few pixels in learned feature maps and realize mask-aware continuous facial attribute manipulation. Gu et al [9] propose an end-to-end framework to learn conditional GANs guided by semantic masks, enabling regional facial style transfer.…”

Section: Facial Image Manipulation With Gansmentioning

confidence: 99%

Towards harmonized regional style transfer and manipulation for facial images

et al. 2023

View full text Add to dashboard Cite

Regional facial image synthesis conditioned on a semantic mask has achieved great attention in the field of computational visual media. However, the appearances of different regions may be inconsistent with each other after performing regional editing. In this paper, we focus on harmonized regional style transfer for facial images. A multi-scale encoder is proposed for accurate style code extraction. The key part of our work is a multi-region style attention module. It adapts multiple regional style embeddings from a reference image to a target image, to generate a harmonious result. We also propose style mapping networks for multi-modal style synthesis. We further employ an invertible flow model which can serve as mapping network to fine-tune the style code by inverting the code to latent space. Experiments on three widely used face datasets were used to evaluate our model by transferring regional facial appearance between datasets. The results show that our model can reliably perform style transfer and multi-modal manipulation, generating output comparable to the state of the art.

show abstract

Mask-aware photorealistic facial attribute manipulation

Cited by 19 publications

References 29 publications

Multi3D: 3D-aware multimodal image synthesis

Multi3D: 3D-aware multimodal image synthesis

Deepfakes Generation and Detection: A Short Survey

Towards harmonized regional style transfer and manipulation for facial images

Contact Info

Product

Resources

About