Region-Aware Face Swapping

Xu, Chao; Zhang, Jiangning; Miao, Hongchen; Qian, He; Yi, Zili; Liu, Yong

doi:10.1109/cvpr52688.2022.00748

Cited by 20 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to [NKH22] and [XZH*22], we quantitatively compare our method to other face‐swapping methods using the FID score, which correlates with human perception the quality of generated images [HRU*17]. We use the FID score to judge the visual quality of the swaps (generated images) when compared to a distribution of real images displaying the source identity.…”

Section: Methodsmentioning

confidence: 99%

“…[LWXS22] propose an end‐to‐end framework where attributes and identity are disentangled by dedicated encoders. In [XZH*22] a region‐aware face swapping network based on GAN inversion is presented. It generates high‐resolution and identity consistent swaps, although due to the StyleGAN2 prior, the method fails to handle difficult face poses.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Otto

Naruniec

Helminger

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Face swapping is the process of applying a source actor's appearance to a target actor's performance in a video. This is a challenging visual effect that has seen increasing demand in film and television production. Recent work has shown that data‐driven methods based on deep learning can produce compelling effects at production quality in a fraction of the time required for a traditional 3D pipeline. However, the dominant approach operates only on 2D imagery without reference to the underlying facial geometry or texture, resulting in poor generalization under novel viewpoints and little artistic control. Methods that do incorporate geometry rely on pre‐learned facial priors that do not adapt well to particular geometric features of the source and target faces. We approach the problem of face swapping from the perspective of learning simultaneous convolutional facial autoencoders for the source and target identities, using a shared encoder network with identity‐specific decoders. The key novelty in our approach is that each decoder first lifts the latent code into a 3D representation, comprising a dynamic face texture and a deformable 3D face shape, before projecting this 3D face back onto the input image using a differentiable renderer. The coupled autoencoders are trained only on videos of the source and target identities, without requiring 3D supervision. By leveraging the learned 3D geometry and texture, our method achieves face swapping with higher quality than when using off‐the‐shelf monocular 3D face reconstruction, and overall lower FID score than state‐of‐the‐art 2D methods. Furthermore, our 3D representation allows for efficient artistic control over the result, which can be hard to achieve with existing 2D approaches.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Otto

Naruniec

Helminger

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Face Swapping. The topic of face swapping has received significant attention in research and is highly relevant, as evidenced by the large body of works dedicated to it [55,43,57,14,83,76,77]. However, it presents inherent and important differences compared to the topic of face anonymization/de-identification.…”

Section: Related Workmentioning

confidence: 99%

Disguise without Disruption: Utility-Preserving Face De-Identification

Cai¹,

Gao²,

Planche³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the increasing ubiquity of cameras and smart sensors, humanity is generating data at an exponential rate. Access to this trove of information, often covering yetunderrepresented use-cases (e.g., AI in medical settings) could fuel a new generation of deep-learning tools. However, eager data scientists should first provide satisfying guarantees w.r.t. the privacy of individuals present in these untapped datasets. This is especially important for images or videos depicting faces, as their biometric information is the target of most identification methods. While a variety of solutions have been proposed to de-identify such images, they often corrupt other non-identifying facial attributes that would be relevant for downstream tasks.In this paper, we propose Disguise, a novel algorithm to seamlessly de-identify facial images while ensuring the usability of the altered data. Unlike prior arts, we ground our solution in both differential privacy and ensemble-learning research domains. Our method extracts and swaps depicted identities with fake ones, synthesized via variational mechanisms to maximize obfuscation and non-invertibility; while leveraging the supervision from a mixture-of-experts to disentangle and preserve other utility attributes. We extensively evaluate our method on multiple datasets, demonstrating higher de-identification rate and superior consistency than prior art w.r.t. various downstream tasks.

show abstract

“…Recently, several studies using the pre-trained StyleGAN [9,29,32,33,36] have proposed to manipulate the latent codes of images for the purpose of facial content manipulation. The GAN Inversion found the latent code of the image and operated on it to modify the image.One is the optimization-based approach of optimizing the latent code to minimize the error for the given image.…”

Section: Latent Space Manipulationmentioning

confidence: 99%

End-to-end Face-swapping via Adaptive Latent Representation Learning

Lin¹,

Hu²,

Shen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Region-Aware Face Swapping

Cited by 20 publications

References 31 publications

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Disguise without Disruption: Utility-Preserving Face De-Identification

End-to-end Face-swapping via Adaptive Latent Representation Learning

Contact Info

Product

Resources

About