A lightweight 3D convolutional neural network for deepfake detection

With the development of media convergence, information acquisition is no longer limited to traditional media, such as newspapers and televisions, but more from digital media on the Internet, where media contents should be under supervision by platforms. At present, the media content analysis technology of Internet platforms relies on deep neural networks (DNNs). However, DNNs show vulnerability to adversarial examples, which results in security risks. Therefore, it is necessary to adequately study the internal mechanism of adversarial examples to build more effective supervision models. When coming to practical applications, supervision models are mostly faced with black‐box attacks, where cross‐model transferability of adversarial examples has attracted increasing attention. In this paper, to improve the transferability of adversarial examples, we propose an attention‐guided transformation‐invariant adversarial attack method, which incorporates an attention mechanism to disrupt the most distinctive features and simultaneously ensures adversarial attack invariance under different transformations. Specifically, we dynamically weight the latent features according to an attention mechanism and disrupt them accordingly. Meanwhile, considering the lack of semantics in low‐level features, high‐level semantics are introduced as spatial guidance to make low‐level feature perturbations concentrate on the most discriminative regions. Moreover, since the attention heatmaps may vary significantly across different models, a transformation‐invariant aggregated attack strategy is proposed to alleviate overfitting to the proxy model attention. Comprehensive experimental results show that the proposed method can significantly improve the transferability of adversarial examples.

Section: Layerwise Attention-guided Adversarial Attackmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Attention‐guided transformation‐invariant attack for black‐box adversarial examples

Zhu

Dai

et al. 2022

“…With the superior performance of deep learning in a wide range of computer vision tasks, 8–14 some recent work based on convolutional neural networks (CNNs) is continuing to explore the potential applications in image manipulation detection 15,16 . The existing methods can be mainly divided into the following two categories: (1) Binary classification detection.…”

Section: Introductionmentioning

confidence: 99%

PR‐NET: Progressively‐refined neural network for image manipulation localization

Shi

Chang

Chen

et al. 2022

Current deep learning‐based image manipulation localization methods achieve impressive performance when rich spatial features and information are fully utilized. However, most of them suffer from the irrelevance of semantic awareness when identifying various manipulation categories. This leads to false alarms on recognizing forged regions. In this paper, we propose a Progressively‐Refined Neural Network (PR‐Net), to localize the tampered regions progressively under a coarse‐to‐fine workflow. Specifically, PR‐Net is composed of a Feature Extractor (FE) that captures feature intrinsic correlations and a Mask Generation Module (MGM) with three refining generators. The FE takes a CNN to extract the image features and introduces an attention mechanism Convolution Block Attention Module (CBAM) to suppress the image content and guide the extractor in exploring the inconsistencies between the manipulated and authentic regions. The MGM comprises three generators where the Coarse Mask RR‐Generator generates a localization result roughly, the Candidate Mask RR‐Generator generates a possible tampered region according to the rough localization measure, and the Fine Mask RR‐Generator produces the final prediction of manipulated regions. We also utilize the Rotated Residual (RR) structure to suppress the image content during the generative process. The extensive experimental results on four benchmark data sets (NIST16, COVER, CASIA v1.0, and In‐The‐Wild) demonstrate the superior performance of PR‐Net compared with the state‐of‐the‐art methods in localizing the manipulated regions.

“…Pun et al 30 took the contextual super‐pixel blocks to assist the classification of the blocks. Furthermore, a line of research focuses on image manipulation localization , which aims to localize the tampered regions 31–33 …”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, a line of research focuses on image manipulation localization, which aims to localize the tampered regions. [31][32][33] The core hypothesis of the image manipulation localization methods is that any of the manipulation operations would leave some abnormal traces. [34][35][36][37] Some localization methods capture the manipulation anomalies by using manually constructed features, such as resampling, 37 noise, 38 edge, 39,40 and so forth.…”

Section: Introductionmentioning

confidence: 99%

MC‐Net: Learning mutually‐complementary features for image manipulation localization

Shen

Lyu

et al. 2022

Deep learning has become an emerging technical for image manipulation localization, which can automatically recognize abnormal traces caused by manipulation. However, as manipulations mainly happens in the foreground regions, these methods largely focus on the foreground contents and neglect the background, which contain complementary signal for fully understanding the image and are meaningful for manipulation localization. We propose a Mutually‐Complementary Network (MC‐Net), which is a two‐branch network to operate the foreground and background features, respectively. To distill complementary signals from the features, we propose a mutual attentive module composed of self‐feature attentive, and cross‐feature attentive components to advance the communication across the foreground and background branches. Extensive qualitative and quantitative experiments demonstrate that our proposed MC‐Net distinctly improves the prediction of foreground and background, obtains consistent performance increments on four benchmark data sets, and significantly outperforms the state‐of‐the‐art methods.