End-to-End Reconstruction-Classification Learning for Face Forgery Detection

Cao, Junyi; Ma, Chao; Yao, Taiping; Shen, Chen; Ding, Shouhong; Yang, Xiaokang

doi:10.1109/cvpr52688.2022.00408

Cited by 88 publications

(34 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To tackle the malicious use of face swapping, numerous detection methods have been proposed. Most existing works model face swapping detection as a binary classification problem and focus on designing better the features classified on [8], [10], [11], [13], [16], [36], the classifier network [7], [9], [12], [15], [37], [38] or the training policies [14], [39], [40] to improve the accuracy and generalization. For example, low-level artifacts are used in early detection methods, such as face warping artifacts [8].…”

Section: Related Workmentioning

confidence: 99%

“…As shown in Table 2, our approach outperforms the competitors on most manipulations. MultiAtt [12], RECCE [15] and DCL [39] are among the state-of-the-art deepfake detection based on binary classifiers. When trained on Deepfakes and tested on FaceSwap, the AUC gaps are larger than 25%.…”

Section: Cross-manipulation Performancementioning

confidence: 99%

“…The result AUC is reported in table 3. MultiAtt [12] and RECCE [15] represent previous detection methods based on low-level features like textures. They are trained on FF++(c23) and evaluated on different compression levels.…”

Section: Cross-quality Performancementioning

confidence: 99%

“…To tackle the malicious application of face swapping, various detection methods have been proposed. A common way is to model face swapping detection as a binary classification task and collect existing face forgery images to train the classifier [7], [8], [9], [10], [11], [12], [13], [14], [15], [16]. Although these approaches achieve high detection accuracy on benchmark datasets, they still struggle to meet some real-world demands.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

Duan¹,

Zhang²,

Yu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, face swapping has been developing rapidly and achieved a surprising reality, raising concerns about fake content. As a countermeasure, various detection approaches have been proposed and achieved promising performance. However, most existing detectors struggle to maintain performance on unseen face swapping methods and low-quality images. Apart from the generalization problem, current detection approaches have been shown vulnerable to evasion attacks crafted by detection-aware manipulators. Lack of robustness under adversary scenarios leaves threats for applying face swapping detection in real world. In this paper, we propose a novel face swapping detection approach based on face identification probability distributions, coined as IdP FSD, to improve the generalization and robustness. IdP FSD is specially designed for detecting swapped faces whose identities belong to a finite set, which is meaningful in real-world applications. Compared with previous general detection methods, we make use of the available real faces with concerned identities and require no fake samples for training. IdP FSD exploits face swapping's common nature that the identity of swapped face combines that of two faces involved in swapping. We reflect this nature with the confusion of a face identification model and measure the confusion with the maximum value of the output probability distribution. What's more, to defend our detector under adversary scenarios, an attention-based finetuning scheme is proposed for the face identification models used in IdP FSD. Extensive experiments show that the proposed IdP FSD not only achieves high detection performance on different benchmark datasets and image qualities but also raises the bar for manipulators to evade the detection.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Cross-manipulation Performancementioning

confidence: 99%

Section: Cross-quality Performancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

Duan¹,

Zhang²,

Yu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Some works [13] [17] utilized auxiliary supervision such as blending boundaries or forged masks. Cao et al [16] presented a reconstruction-classification learning method that mines the common features of genuine faces. Sun et al [15] proposed a novel Dual Contrastive Learning to specially construct positive and negative data pairs and perform contrastive learning at different granularities.…”

Section: Introductionmentioning

confidence: 99%

An Attention-Erasing Stripe Pyramid Network for Face Forgery Detection

Duan

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

ace forgery detection aims to distinguish between real and fake facial images or videos by identifying manipulated or forged visual media. The main challenge in face forgery detection is achieving high model generalization ability, i.e., satisfactory performance under cross-database scenarios where the training and testing datasets are from different forgery methods. To achieve this goal, this paper presents an Attention-Erasing Stripe Pyramid Network (ASPNet) to utilize high-frequency noises and exploit both the RGB and fine-grained frequency clues. First, since separately extracting features from different scales and granularities will ignore their complementarity, we employ a Stripe Pyramid Block (SPB) to learn multi-scale and multi-granularity features simultaneously. Second, to make the model to focus on useful information and suppress noise, a Two-Stage Attention Block (TSAB) is introduced by combining spatial attention and channel attention to filter out the pixel-wise and channel-wise noise in the learned feature maps. Finally, to dynamically guide the model to pay attention to different areas of the human face, an Attention Erasing (AE) scheme is adopted by randomly erasing units in attention maps. Sufficient experiments demonstrate that ASPNet has superior performance than F3-Net on the FaceForensics++ dataset. The Area Under the Receiver Operating Characteristic Curve (AUC) and the Accuracy (ACC) of our model reach 77.4% and 70.85% respectively, whichare improved by 0.83% and 1.28% compared with F3-Net. Our code is available at: https://github.com/NWPU-Zwu.

show abstract

Hierarchical Contrastive Inconsistency Learning for Deepfake Video Detection

Yao

Chen

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

End-to-End Reconstruction-Classification Learning for Face Forgery Detection

Cited by 88 publications

References 45 publications

Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

An Attention-Erasing Stripe Pyramid Network for Face Forgery Detection

Hierarchical Contrastive Inconsistency Learning for Deepfake Video Detection

Contact Info

Product

Resources

About