Leveraging High-Frequency Components for Deepfake Detection

Abstract

In the past years, RGB-based deepfake detection has shown notable progress thanks to the development of effective deep neural networks. However, the performance of deepfake detectors remains primarily dependent on the quality of the forged content and the level of artifacts introduced by the forgery method. To detect these artifacts, it is often necessary to separate and analyze the frequency components of an image. In this context, we propose to utilize the high-frequency components of color images by introducing an end-to-end trainable module that (a) extracts features from high-frequency components and (b) fuses them with the features of the RGB input. The module not only exploits the high-frequency anomalies present in manipulated images but also can be used with most RGB-based deepfake detectors. Experimental results show that the proposed approach boosts the performance of state-of-the-art networks, such as XceptionNet and EfficientNet, on a challenging deepfake dataset. I. INTRODUCTIONDeepfakes are images and videos that seem genuine to the human eyes, whereas, in reality, they are either entirely or partially generated by an artificial intelligence algorithm. Deepfakes appeared in 2017 as adult forged content, depicting faces that were swapped with celebrities' faces [1]. As a technology, deepfakes have creative applications in movie post-production, dubbing, productive education, and identity anonymization. Nevertheless, they remain a significant threat to the public order and international peace 1 , especially with the virality of social media 2 .Consequently, developing automated deepfake detection tools has become a pressing matter. Deepfake detection started growing alongside the development of deepfake generation methods and open-source software like Face-Swap 3 , FakeApp 4 and DeepFaceLab [6]. Researchers started building image and video databases of fake content, focusing on key properties, such as visual quality, level of artifacts, and setup diversity. Most of deepfake detectors operate on RGB data, as it is the most abundant form [5]. These detectors can be artifact-specific or undirected [7]. In the first case, they try to find particular anomalies produced by the deepfake generation methods. Such irregularities can manifest as inconsistencies in the noise level, the color, the 1 https://www.theguardian.com/world/2021/apr/22/ european-mps-targeted-by-deepfake-video-calls-imitating-russian-opposition 2 https://www.bbc.com/news/technology-49961089 3 https:

Multi-Label Deepfake Classification

“…The subtlety of these forgeries makes their distinction from authentic images increasingly challenging. Given this threat, many efforts have been dedicated to developing deepfake detection techniques [1]- [3]. Typically, these approaches formalize the problem of deepfake detection as a binary classification [2]- [8].…”

“…In fact, an image predicted as fake can be produced by one or multiple manipulations. In existing face editing software, such as FaceTune 1 , it is common for the same image to undergo several edits, which we refer to as stacked manipulations or multi-step operations, as illustrated in Fig. 1(a).…”