Face X-Ray for More General Face Forgery Detection

Abstract

In this paper we propose a novel image representation called face X-ray for detecting forgery in face images. The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources. It does so by showing the blending boundary for a forged image and the absence of blending for a real image. We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image. For this reason, face X-ray provides an effective way for detecting forgery generated by most existing face manipulation algorithms. Face X-ray is general in the sense that it only assumes the existence of a blending step and does not rely on any knowledge of the artifacts associated with a specific face manipulation technique. Indeed, the algorithm for computing face X-ray can be trained without fake images generated by any of the state-of-the-art face manipulation methods. Extensive experiments show that face Xray remains effective when applied to forgery generated by unseen face manipulation techniques, while most existing face forgery detection algorithms experience a significant performance drop.

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

“…To detect the Deepfake images or videos, most of the previous works are based on deep learning methods, which can be categorized into two detection methods: CNN-based methods [10,13,15,16,[20][21][22] and RCNN-based methods [11,17,18].…”

“…In addition, the loss of label classifier G y is also minimized. e overall loss function of DANN can be formalized as [20] CNN Multitask ForensicTransfer [21] CNN Multitask Face X-Ray [22] CNN Multitask Qian et al [24] CNN Frequency Li et al [11] CNN + LSTM Handcrafted Guera et al [17] CNN + LSTM RGB Chen et al [18] CNN + LSTM RGB Yang et al [12] SVM Handcrafted FakeCatcher [25] SVM Handcrafted…”

“…Unfortunately, Li et al [19] found that most of the Deepfake detection methods trained and tested on specific datasets can achieve satisfactory performance, but their performances are significantly reduced when the methods are tested on cross-domain datasets, indicating that these methods are overfitting on a specific dataset. To improve the generalization ability of the methods on cross-domain datasets, multitask learning approaches [20][21][22] were introduced for Deepfake detection. Specifically, Nguyen et al [20] developed a multitask learning approach to simultaneously perform classification, reconstruction, and segmentation of manipulated facial images.…”

“…Specifically, Nguyen et al [20] developed a multitask learning approach to simultaneously perform classification, reconstruction, and segmentation of manipulated facial images. Cozzolino et al [21] proposed the "ForensicTransfer" by combining classification and reconstruction, while Li et al [22] proposed the "Face X-Ray" to detect Deepfake videos based on blending boundaries by combining classification and segmentation. However, those methods still need to improve the performance of the cross-dataset evaluation because they tend to train the classifier on a single small-scale dataset (i.e., FaceForensics++ [16] dataset), which is difficult to be generalized to other unseen datasets generated by using unseen Deepfake manipulation methods.…”

Attentive Semantic Exploring for Manipulated Face Detection

“…They are not well equipped to detect nowaday diverse types of fake images. Scholars try to tackle diverse types of fake face images with multifarious ideas in recent studies [21,22,23,24,25,26,27]. For instance, [21] proposes an auto-encoder-based model to detect manipulated face images.…”