GLFF: Global and Local Feature Fusion for Face Forgery Detection

Abstract

With the rapid development of deep generative models (such as Generative Adversarial Networks and Autoencoders), AI-synthesized images of the human face are now of such high quality that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images from seen models or on images without real-world postprocessing, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multiscale global features from the whole image with refined local features from informative patches for face forgery detection. GLFF fuses information from two branches: the global branch to extract multi-scale semantic features and the local branch to select informative patches for detailed local artifacts extraction. Due to the lack of a face forgery dataset simulating real-world applications for evaluation, we further create a challenging face forgery dataset, named DeepFakeFaceForensics (DF 3 ), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world scenarios. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF 3 dataset and three other open-source datasets.

A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods

“…In the same year, Jia et al [65] developed a face forgery detection method that utilized a fusion of global and local features. Face forgery detection aims to identify instances in which a person's face has been manipulated or replaced with another person's face.…”