ADD: Attention-Based DeepFake Detection Approach

Khormali, Aminollah; Yuan, J.S.

doi:10.3390/bdcc5040049

Cited by 23 publications

(22 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Du et al [13] proposed a deepfake detection method from a fine-grained visual classification angle that is built using an auto-encoder architecture. Furthermore, Khormali and Yuan [18] have presented an attention-based deepfake detection approach utilizing two different modules, i.e., Face close-up and Face Shut-off, to force the model to extract more discriminative information from other parts of the facial region. Quan et al [41] presented a progressive transfer learning algorithm to tackle face spoofing attacks using only a limited number of training samples.…”

Section: Related Workmentioning

confidence: 99%

“…One solution to improve the transformer model's capability in capturing sensitive information is to pay more attention to discriminative patches within the training and inference phases, as depicted in Figure 2. In addition, attention-based mechanisms have demonstrated their strong capability in improving the performance of traditional CNN-based deepfake detection models [18]. Therefore, inspired by [53], a patch selection mechanism based on attention weights is l in this study.…”

Section: Attention-based Patch Selectionmentioning

confidence: 99%

“…This figure also reveals another critical point: although most deepfake detection approaches perform well on relatively more straightforward datasets, i.e., FaceForensics++, their performance is still far from perfect on more challenging and real-world datasets, i.e., WildDeepfake. [46] 99.64 F3-Net [73] 65.1 ADD-Xception [18] 79.23 RNN [74] 83.10 PPA [75] 83.1 DefakeHop [6] 90.5 FakeCatcher [15] 91.5 ATS-DE [7] 97.8 ADD-ResNet [18] 98. MesoNet-4 [34] ADDNet-3D [75] MesoNet-inception [34] XceptionNet [65] ADDNet-2D [75] ADD-Xception [55] DFDT (Ours) ACC (%)…”

Section: Intra-dataset Evaluationmentioning

confidence: 99%

“…Typical components of most state-of-the-art deepfake detection approaches are convolutional neural networks, and facial regions cropped out of an entire image [14][15][16]. Unique characteristics of the convolutional operator in deep CNNs have enabled them to demonstrate strong capabilities on capturing minor visual artifacts, yielding decent detection results [7,17,18]. Although CNNs have proven themselves solid candidates for learning local information of the image, they still miss capturing pixels' spatial interdependence due to constrained receptive fields.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

Self Cite

View full text Add to dashboard Cite

The ever-growing threat of deepfakes and large-scale societal implications has propelled the development of deepfake forensics to ascertain the trustworthiness of digital media. A common theme of existing detection methods is using Convolutional Neural Networks (CNNs) as a backbone. While CNNs have demonstrated decent performance on learning local discriminative information, they fail to learn relative spatial features and lose important information due to constrained receptive fields. Motivated by the aforementioned challenges, this work presents DFDT, an end-to-end deepfake detection framework that leverages the unique characteristics of transformer models, for learning hidden traces of perturbations from both local image features and global relationship of pixels at different forgery scales. DFDT is specifically designed for deepfake detection tasks consisting of four main components: patch extraction & embedding, multi-stream transformer block, attention-based patch selection followed by a multi-scale classifier. DFDT’s transformer layer benefits from a re-attention mechanism instead of a traditional multi-head self-attention layer. To evaluate the performance of DFDT, a comprehensive set of experiments are conducted on several deepfake forensics benchmarks. Obtained results demonstrated the surpassing detection rate of DFDT, achieving 99.41%, 99.31%, and 81.35% on FaceForensics++, Celeb-DF (V2), and WildDeepfake, respectively. Moreover, DFDT’s excellent cross-dataset & cross-manipulation generalization provides additional strong evidence on its effectiveness.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Attention-based Patch Selectionmentioning

confidence: 99%

Section: Intra-dataset Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Research since then has evolved by combining CNNs with other architectures such as Recurrent Neural Networks (RNNs) [19], Long Short-Term Memories (LSTMs) [36,47] or Attention heads [14,30,73,80,81,84]. In [3] the authors propose an ensemble of numerous CNN classifiers, based on the popular EfficientNet network [66] in tandem with attention mechanisms and Siamese training with the goal of accurately detecting DeepFakes.…”

Section: Deepfake Detection Approachesmentioning

confidence: 99%

The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild

Baxevanakis¹,

Kordopatis-Zilos²,

Galopoulos³

et al. 2022

Preprint

View full text Add to dashboard Cite

Enabled by recent improvements in generation methodologies, DeepFakes have become mainstream due to their increasingly better visual quality, the increase in easy-to-use generation tools and the rapid dissemination through social media. This fact poses a severe threat to our societies with the potential to erode social cohesion and influence our democracies. To mitigate the threat, numerous DeepFake detection schemes have been introduced in the literature but very few provide a web service that can be used in the wild. In this paper, we introduce the MeVer DeepFake detection service, a web service detecting deep learning manipulations in images and video. We present the design and implementation of the proposed processing pipeline that involves a model ensemble scheme, and we endow the service with a model card for transparency. Experimental results show that our service performs robustly on the three benchmark datasets while being vulnerable to Adversarial Attacks. Finally, we outline our experience and lessons learned when deploying a research system into production in the hopes that it will be useful to other academic and industry teams. CCS CONCEPTS• Information systems → Multimedia information systems; Web services; Data analytics; • Security and privacy → Human and societal aspects of security and privacy.

show abstract

Deepfakes: evolution and trends

et al. 2023

View full text Add to dashboard Cite

This study conducts research on deepfakes technology evolution and trends based on a bibliometric analysis of the articles published on this topic along with six research questions: What are the main research areas of the articles in deepfakes? What are the main current topics in deepfakes research and how are they related? Which are the trends in deepfakes research? How do topics in deepfakes research change over time? Who is researching deepfakes? Who is funding deepfakes research? We have found a total of 331 research articles about deepfakes in an analysis carried out on the Web of Science and Scopus databases. This data serves to provide a complete overview of deepfakes. Main insights include: different areas in which deepfakes research is being performed; which areas are the emerging ones, those that are considered basic, and those that currently have the most potential for development; most studied topics on deepfakes research, including the different artificial intelligence methods applied; emerging and niche topics; relationships among the most prominent researchers; the countries where deepfakes research is performed; main funding institutions. This paper identifies the current trends and opportunities in deepfakes research for practitioners and researchers who want to get into this topic.

show abstract

ADD: Attention-Based DeepFake Detection Approach

Cited by 23 publications

References 54 publications

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild

Deepfakes: evolution and trends

Contact Info

Product

Resources

About