A Transformer-Based DeepFake-Detection Method for Facial Organs

Abstract

Nowadays, deepfake detection on subtle-expression manipulation, facial-detail modification, and smeared images has become a research hotspot. Existing deepfake-detection methods on the whole face are coarse-grained, where the details are missing due to the negligible manipulated size of the image. To address the problems, we propose to build a transformer model for a deepfake-detection method by organ, to obtain the deepfake features. We reduce the detection weight of defaced or unclear organs to prioritize the detection of clear and intact organs. Meanwhile, to simulate the real-world environment, we build a Facial Organ Forgery Detection Test Dataset (FOFDTD), which includes the images of mask face, sunglasses face, and undecorated face collected from the network. Experimental results on four benchmarks, i.e., FF++, DFD, DFDC-P, Celeb-DF, and for FOFDTD datasets, demonstrated the effectiveness of our proposed method.

Local Corner and Motion Key Point Trajectory Extraction for Facial Forgery Identification

“…However, the early works were mainly focused on global features. Specifcally, we notice that forgery detection features are particularly evident in key facial organs such as the eyes, nose, and mouth [5,6,12]. For example, Xue et al [12] found that only using facial organs such as the nose, lips, eyes, eyebrows, and chin can detect deep forgery very well.…”

“…Wang et al [14] proposed a method that fused facial region feature descriptor for forgery determination by extracting feature points of a person's face. Xue et al [12] built a transformer model for a deepfake-detection method by organs to obtain the deepfake features. Yang et al [15] proposed a method for detecting diferences in face textures by amplifying the texture diferences between genuine and fake images and using a bootstrap flter to enhance postprocessing-induced texture artifacts and display the underlying features of the artifacts.…”

A review of deep learning‐based approaches for deepfake content detection

“…The work developed by (Xue et al, 2022) proposes a transformer-based deepfake detection method for facial organs, which can effectively differentiate deepfake media. The authors highlight that deepfake detection on subtle-expression manipulation, facial-detail modification, and smeared images has become a wide research hotspot.…”