Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

Chen, Xu; Zhang, Jiangning; Han, Yuexin; Tian, Guanzhong; Zeng, Xianfang; Tai, Ying; Wang, Yabiao; Wang, Chengjie; Liu, Yong

doi:10.1007/978-3-031-19784-0_4

Cited by 11 publications

(6 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, although slightly affected by Salt and Pepper (95.99%) and Gaussian Blur (97.11%), we successfully maintained state-of-the-art accuracies above 95% for all manipulations. Furthermore, an obvious accuracy decline on UniFace [43] can be observed in Table 2 and Table 4 at 256 × 256 resolutions on both CelebA-HQ [16] and LFW. Besides the challenges caused by cross manipulation, this is mainly due to the setting of accepting only half precision floating-points inputs by UniFace.…”

Section: Cross-dataset Experimentsmentioning

confidence: 95%

“…P common consists of the following manipulations for both training and testing: Dropout, Resize, Jpeg Compression, Gaussian Noise, Salt and Pepper, Gaussian Blur, and Median Blur. P swap includes Sim-Swap [4] solely in the training phase and adopts SimSwap, InfoSwap [8], and UniFace [43] with state-of-the-art performance for cross-manipulation evaluation in the testing phase. Since there are various common manipulations to be considered and they only bring noises without modifying the facial contents, we first pre-trained the model using P common for sufficient iterations.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…Watermark Recovery on Face Swapping. In this paper, the face swapping manipulation pool contains merely SimSwap [4] in the training phase, and we validated the watermark recovery performance for both in-and crossmanipulation settings by introducing other recent state-ofthe-art face swapping algorithms, InfoSwap [8] and Uni-Face [43], in the testing phase. As a result, our model consistently recovers the desired watermark messages at average accuracies of 99.62% and 98.19% for 128 × 128 and 256 × 256 resolutions, respectively, while CIN and MBRS both obtain accuracies around 50% and RootAttr has accuracies below 75% against all synthetic models.…”

Section: Quantitative Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

Implicit Identity Driven Deepfake Face Swapping Detection

Huang,

Wang,

Yang

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Notwithstanding offering convenience and entertainment to society, Deepfake face swapping has caused critical privacy issues with the rapid development of deep generative models. Due to imperceptible artifacts in high-quality synthetic images, passive detection models against face swapping in recent years usually suffer performance damping regarding the generalizability issue. Therefore, several studies have been attempted to proactively protect the original images against malicious manipulations by inserting invisible signals in advance. However, the existing proactive defense approaches demonstrate unsatisfactory results with respect to visual quality, detection accuracy, and source tracing ability. In this study, we propose the first robust identity perceptual watermarking framework that concurrently performs detection and source tracing against Deepfake face swapping proactively. We assign identity semantics regarding the image contents to the watermarks and devise an unpredictable and unreversible chaotic encryption system to ensure watermark confidentiality. The watermarks are encoded and recovered by jointly training an encoder-decoder framework along with adversarial image manipulations. Extensive experiments demonstrate stateof-the-art performance against Deepfake face swapping under both cross-dataset and cross-manipulation settings.

show abstract

Section: Cross-dataset Experimentsmentioning

confidence: 95%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Quantitative Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Implicit Identity Driven Deepfake Face Swapping Detection

Huang,

Wang,

Yang

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Although some general representation learning methods have achieved good performance, the method designed for face images still has better results than the one training on the general dataset [32]. Supervised representation learning methods [21,22,26,30,31] need amounts of manually labeled data and suffer from overfitting problems with a large number of model parameters. Recently some tasks [18,32] use Web text-image pairs as pre-train dataset, which achieve amazing representation learning results.…”

Section: Related Work 21 Facial Representation Learningmentioning

confidence: 99%

Toward High Quality Facial Representation Learning

Wang,

Peng,

Zhang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

Face analysis tasks have a wide range of applications, but the universal facial representation has only been explored in a few works. In this paper, we explore high-performance pre-training methods to boost the face analysis tasks such as face alignment and face parsing. We propose a self-supervised pre-training framework, called Mask Contrastive Face (MCF), with mask image modeling and a contrastive strategy specially adjusted for face domain tasks. To improve the facial representation quality, we use feature map of a pre-trained visual backbone as a supervision item and use a partially pre-trained decoder for mask image modeling. To handle the face identity during the pre-training stage, we further use random masks to build contrastive learning pairs. We conduct the pre-training on the LAION-FACE-cropped dataset, a variants of LAION-FACE 20M, which contains more than 20 million face images from Internet websites. For efficiency pre-training, we explore our framework pre-training performance on a small part of LAION-FACE-cropped and verify the superiority with different pre-training settings. Our model pre-trained with the full pre-training dataset outperforms the state-of-the-art methods on multiple downstream tasks. Our model achieves 0.932 NME 𝑑𝑖𝑎𝑔 for AFLW-19 face alignment and 93.96 F1 score for LaPa face parsing. Code is available at https://github.com/nomewang/MCF.

show abstract

“…In one approach to face swapping, the reconstruction loss between the swapped and target images is employed when the source and target images share the same identity. [5,32,20,35,34,33]. However, applying the reconstruction loss in certain scenarios necessitates the use of identitylabeled image datasets [24,4] or video datasets [23,6].…”

Section: Introductionmentioning

confidence: 99%

RobustSwap: A Simple yet Robust Face Swapping Model against Attribute Leakage

Lee¹,

Kim²,

Park³

et al. 2023

Preprint

View full text Add to dashboard Cite

robustswap.github.io/ Figure 1. Face swapping results of RobustSwap. Our model robustly synthesizes the swapped image from the in-the-wild source and target images, while preserving the target attributes and reflecting source identity. With our model, the target attributes (e.g., eye gazing, hair, background, expression, and head pose) are preserved successfully regardless of source identity images.

show abstract

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

Cited by 11 publications

References 41 publications

Implicit Identity Driven Deepfake Face Swapping Detection

Implicit Identity Driven Deepfake Face Swapping Detection

Toward High Quality Facial Representation Learning

RobustSwap: A Simple yet Robust Face Swapping Model against Attribute Leakage

Contact Info

Product

Resources

About