DeepFake detection algorithm based on improved vision transformer

Heo, Young-Jin; Yeo, Woon-Ha; Kim, Byung-Gyu

doi:10.1007/s10489-022-03867-9

Cited by 42 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Arshed et al [123] applied vision transformers over deepfakes and got excellent results on deepfake images shared over Kaggle. Similar results were observed by Heo et al [124] with deepfake videos. They combine patch-based positioning and vector-concatenated CNN features to interact with all positions to determine the artifact region.…”

Section: Convolutional Neural Network-image Specificsupporting

confidence: 91%

A Survey of Detection and Mitigation for Fake Images on Social Media Platforms

Sharma,

Singh,

Agarwal

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Recently, the spread of fake images on social media platforms has become a significant concern for individuals, organizations, and governments. These images are often created using sophisticated techniques to spread misinformation, influence public opinion, and threaten national security. This paper begins by defining fake images and their potential impact on society, including the spread of misinformation and the erosion of trust in digital media. This paper also examines the different types of fake images and their challenges for detection. We then review the recent approaches proposed for detecting fake images, including digital forensics, machine learning, and deep learning. These approaches are evaluated in terms of their strengths and limitations, highlighting the need for further research. This paper also highlights the need for multimodal approaches that combine multiple sources of information, such as text, images, and videos. Furthermore, we present an overview of existing datasets, evaluation metrics, and benchmarking tools for fake image detection. This paper concludes by discussing future directions for fake image detection research, such as developing more robust and explainable methods, cross-modal fake detection, and the integration of social context. It also emphasizes the need for interdisciplinary research that combines computer science, digital forensics, and cognitive psychology experts to tackle the complex problem of fake images. This survey paper will be a valuable resource for researchers and practitioners working on fake image detection on social media platforms.

show abstract

Section: Convolutional Neural Network-image Specificsupporting

confidence: 91%

A Survey of Detection and Mitigation for Fake Images on Social Media Platforms

Sharma,

Singh,

Agarwal

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Its remarkable capability to capture extensive long-range and global contextual information has stimulated the adaptation of transformer to computer vision (CV) tasks [15,32] and derived a series of architectures called vision transformers (ViTs). For face forgery detection, some works have [17,33] directly reshaped the features extracted by CNN into a series of low-dimensional patches and passed them to VITs encoder, achieving certain levels of generalization performance. M2TR [34] operated patches of different sizes to model a multiscale transformer.…”

Section: 2mentioning

confidence: 99%

Face Forgery Detection with Long-Range Noise Features and Multilevel Frequency-Aware Clues

Zhao,

Jin,

Gao

et al. 2024

IET Biometrics

View full text Add to dashboard Cite

The widespread dissemination of high-fidelity fake faces created by face forgery techniques has caused serious trust concerns and ethical issues in modern society. Consequently, face forgery detection has emerged as a prominent topic of research to prevent technology abuse. Although, most existing face forgery detectors demonstrate success when evaluating high-quality faces under intra-dataset scenarios, they often overfit manipulation-specific artifacts and lack robustness to postprocessing operations. In this work, we design an innovative dual-branch collaboration framework that leverages the strengths of the transformer and CNN to thoroughly dig into the multimodal forgery artifacts from both a global and local perspective. Specifically, a novel adaptive noise trace enhancement module (ANTEM) is proposed to remove high-level face content while amplifying more generalized forgery artifacts in the noise domain. Then, the transformer-based branch can track long-range noise features. Meanwhile, considering that subtle forgery artifacts could be described in the frequency domain even in a compression scenario, a multilevel frequency-aware module (MFAM) is developed and further applied to the CNN-based branch to extract complementary frequency-aware clues. Besides, we incorporate a collaboration strategy involving cross-entropy loss and single center loss to enhance the learning of more generalized representations by optimizing the fusion features of the dual branch. Extensive experiments on various benchmark datasets substantiate the superior generalization and robustness of our framework when compared to the competing approaches.

show abstract

“…Moreover, compared to state-of-the-art solutions, the method achieved the highest mean accuracy in the four face manipulation strategies of the FaceForensics++ dataset. (Heo et al, 2023) proposed a DeepFake detection using a Vision Transformer Model, which has indicated good performance in recent image classifications and combined CNN and patch-embedding features during the input stage. The Robust Vision Transformer Model has shown efficiency compared with EfficientNet as the state-of-the-art model, which consists of a 2D CNN network.…”

Section: Cnn+grumentioning

confidence: 99%

A review of deep learning‐based approaches for deepfake content detection

Passos,

Jodas,

Costa

et al. 2024

Expert Systems

View full text Add to dashboard Cite

Recent advancements in deep learning generative models have raised concerns as they can create highly convincing counterfeit images and videos. This poses a threat to people's integrity and can lead to social instability. To address this issue, there is a pressing need to develop new computational models that can efficiently detect forged content and alert users to potential image and video manipulations. This paper presents a comprehensive review of recent studies for deepfake content detection using deep learning‐based approaches. We aim to broaden the state‐of‐the‐art research by systematically reviewing the different categories of fake content detection. Furthermore, we report the advantages and drawbacks of the examined works, and prescribe several future directions towards the issues and shortcomings still unsolved on deepfake detection.

show abstract

DeepFake detection algorithm based on improved vision transformer

Cited by 42 publications

References 26 publications

A Survey of Detection and Mitigation for Fake Images on Social Media Platforms

A Survey of Detection and Mitigation for Fake Images on Social Media Platforms

Face Forgery Detection with Long-Range Noise Features and Multilevel Frequency-Aware Clues

A review of deep learning‐based approaches for deepfake content detection

Contact Info

Product

Resources

About