Fusing Global and Local Features for Generalized AI-Synthesized Image Detection

Abstract

With the development of the Generative Adversarial Networks (GANs) and DeepFakes, AI-synthesized images are now of such high quality that humans can hardly distinguish them from real images. It is imperative for media forensics to develop detectors to expose them accurately. Existing detection methods have shown high performance in generated images detection, but they tend to generalize poorly in the real-world scenarios, where the synthetic images are usually generated with unseen models using unknown source data. In this work, we emphasize the importance of combining information from the whole image and informative patches in improving the generalization ability of AI-synthesized image detection. Specifically, we design a two-branch model to combine global spatial information from the whole image and local informative features from multiple patches selected by a novel patch selection module. Multi-head attention mechanism is further utilized to fuse the global and local features. We collect a highly diverse dataset synthesized by 19 models with various objects and resolutions to evaluate our model. Experimental results demonstrate the high accuracy and good generalization ability of our method in detecting generated images.

GLFF: Global and Local Feature Fusion for Face Forgery Detection

“…We use 5 state-of-the-art generated image detection models as the baselines, of which 3 models are based on spatial information: CNN aug [1], No down [10], and PSM [12] and 2 are based on spectral information: GAN-DCT [18] and BeyongtheSpectrum [2]. We follow the settings in their paper and train them on our training dataset from scratch except for No down, in which the training code was not provided so we used their released pre-trained weights.…”

“…These detectors have achieved good performance on images from seen generation models, where the testing samples are generated using the same model structure or similar source data as the training samples. However, their performance tends to decrease considerably in open-world applications, where the testing images come from a different domain such as being generated by unknown models using unseen source data [12], [13]. When facing such diverse and unknown testing samples, the features learned by current detection models can hardly conduct robust and accurate detection [13].…”

“…Our method achieves outstanding generalization ability on the DF 3 dataset and several public datasets. A preliminary version of this work was in [12], in which we proposed a two-branch model to combine global information and subtle local features to detect diverse types of AIsynthesized images. This work extends on that in two ways: 1) In the global branch, inspired by recent work [9] revealing that slight artifacts caused by forgery methods tend to be preserved in the shallow features, we extract and fuse lowlevel features with high-level semantic features for more finegrained feature learning.…”

Global–Local Facial Fusion Based GAN Generated Fake Face Detection

“…With the deepening of research, some current methods utilize the local features of the face to enhance the global features to obtain more acceptable results. Ju et al [ 18 ] proposed a two-branch model to combine global spatial information from the whole image and local features from multiple patches selected by a novel patch selection module. Zhao et al [ 20 ] proposed a method containing global information and local information.…”

“…In recent years, some approaches [ 18 , 19 , 20 ] have synthesized the local and global features to detect forgeries. The above methods focus on the point that GAN-generated faces are more likely to produce traces in local regions, so they strengthen the forgery detection in the local area and use it to supply the global detection results.…”