Dual Contrastive Learning for General Face Forgery Detection

Abstract

With various facial manipulation techniques arising, face forgery detection has drawn growing attention due to security concerns. Previous works always formulate face forgery detection as a classification problem based on cross-entropy loss, which emphasizes category-level differences rather than the essential discrepancies between real and fake faces, limiting model generalization in unseen domains. To address this issue, we propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which specially constructs positive and negative paired data and performs designed contrastive learning at different granularities to learn generalized feature representation. Concretely, combined with the hard sample selection strategy, Inter-Instance Contrastive Learning (Inter-ICL) is first proposed to promote task-related discriminative features learning by especially constructing instance pairs. Moreover, to further explore the essential discrepancies, Intra-Instance Contrastive Learning (Intra-ICL) is introduced to focus on the local content inconsistencies prevalent in the forged faces by constructing local region pairs inside instances. Extensive experiments and visualizations on several datasets demonstrate the generalization of our method against the state-of-the-art competitors. Our Code is available at https://github.com/Tencent/TFace.git.

UIA-ViT: Unsupervised Inconsistency-Aware Method Based on Vision Transformer for Face Forgery Detection

“…Based on this idea, we can force the model to focus on intra-inconsistency using pseudo annotations. In addition, different from the previous works [27,1] which specially design a module to obtain the consistency-related representation, we find that Vision Transformer [5] naturally provides the consistency representation from the attention map among patch embeddings, thanks to their self-attention mechanism. Therefore, we apply it to build the detection network and propose two key components: UPCL (Unsupervised Patch Consistency Learning) and PCWA (Progressive Consistency Weighted Assemble).…”

“…Some works focused on inevitable procedures in forgery, such as affine warping (DSP-FWA [14]) and blending (Face X-ray [12]). While others observed that certain type of inconsistency exists in different kinds of forgery videos, such as temporal inconsistency (LipForensics [7], FTCN+TT [35]) and intra-frame inconsistency (Local-related [1], PCL+I2G [34], Dual [27]). However, in order to learn corresponding artifacts or inconsistency cues, Face X-ray [12] and PCL+I2G [34] try to generate the large-scale datasets with annotated forged location for their pixel-level supervised learning.…”

“…The generation process is time consuming and cannot capture the properties of forgery regions. Local-related [1] and DCL [27] try to generate annotated forged location labels by subtracting forgery image with its corresponding real image. However, these paired images are usually unavailable, especially in the real-world scenes.…”

“…are quite different. To improve the generalization of detection, common forged artifacts or inconsistency produced by face manipulation techniques are explored by recent methods, such as eye blinking frequency [13], affine warping [14], image blending [12], temporal inconsistency [35,27], intra-frame inconsistency [34,1] and so on. Among them, intra-frame inconsistency has been proved to be able to effectively improve the generalization performance of the detection, since the common face forgery strategy (manipulation and blending) causes the inconsistency between the forged region and the original background.…”

“…Although it can produce accurate pixel-level location annotations, models can not capture the properties of forgery regions, since the generated data is only composed of real images. Other works [1,27] attempt to generate annotated forged location labels by subtracting forgery image with its corresponding real image. However, these paired images are usually unavailable, especially in the real-world scenes.…”