3D CNN Architectures and Attention Mechanisms for Deepfake Detection

Roy, Ritaban; Joshi, Indu; Das, Abhijit; Dantcheva, Antitza

doi:10.1007/978-3-030-87664-7_10

Cited by 16 publications

(6 citation statements)

References 48 publications

(73 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Face)], 83 MesoNet, 42 Bayer et al. method, 47 EfficientNet-b5, 24 Inception ResNet, 84 Conv-LSTM, 52 CViT, 85 3D ResNet, 86 RECCE, 87 GFFD, 88 RFM, 89 DCL, 90 and FedForgery 91 . For this comparison, we consider the SIFT descriptor, which obtained the best performance using the face component.…”

Section: Methodsmentioning

confidence: 99%

On the effectiveness of handcrafted features for deepfake video detection

Kaddar,

Fezza,

Hamidouche

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

Recent developments in advanced generative deep learning techniques have led to considerable progress in deepfake technology. CNN-based deepfake detection approaches have demonstrated superior performance. The ability to learn meaningful representations generated by convolutional multilayer nonlinear structures is the key to success. However, the black-box nature of such approaches has been a major concern for exploring hidden and complex characteristics as well as potential limitations of CNN-based models. To gain insights into the scope of the deepfake detection task, we investigate the effectiveness of handcrafted feature-based methods for deepfake video detection. First, we experiment with six top-performing handcrafted descriptors to extract the discriminating image features and then train SVMs on the extracted features to learn a suitable model. We also study the effect of selecting specific facial components on the detection performance. Specifically, we consider features extracted from the left eye, right eye, mouth, and entire face. Moreover, we propose a combination of these features and highlight the importance of this combination in terms of detection performance. Experimental results show that the SIFT feature descriptor achieves the best performance on deepfake videos generated by the neural texture technique, with a detection accuracy of 83.50%, which is better than deep learning-based methods. This is in contrast to the conventional understanding that deep learning methods systematically outperform handcrafted feature-based approaches. In addition, the obtained results on the FaceForensics++ dataset highlight the benefit of using some facial components to further boost the detection performance. Moreover, motivated by the effectiveness of the LBPTOP and SIFT in the deepfake detection task, we combined the LBPTOP and SIFT to best characterize the specific spatiotemporal inconsistencies commonly found in fake videos for boosting deepfake detection performance. Finally, we show the strengths and weaknesses of methods based on handcrafted features for deepfake detection and provide directions for future research.

show abstract

Section: Methodsmentioning

confidence: 99%

On the effectiveness of handcrafted features for deepfake video detection

Kaddar,

Fezza,

Hamidouche

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Furthermore, they used EfficientNet Variant B4 for the detection of fake images or videos. Roy et al (2022) first trained attention on the dataset to get the most prominent features of the video and then used I3D, 3D ResNet, and 3D ResNeXt to detect deepfakes. Kolagati, Priyadharshini & Mary Anita Rajam (2022) used the multi-layer perceptron to learn the difference between fake and real videos.…”

Section: Methodsmentioning

confidence: 99%

Deepfake attack prevention using steganography GANs

Noreen¹,

Muneer²,

Gillani³

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Background Deepfakes are fake images or videos generated by deep learning algorithms. Ongoing progress in deep learning techniques like auto-encoders and generative adversarial networks (GANs) is approaching a level that makes deepfake detection ideally impossible. A deepfake is created by swapping videos, images, or audio with the target, consequently raising digital media threats over the internet. Much work has been done to detect deepfake videos through feature detection using a convolutional neural network (CNN), recurrent neural network (RNN), and spatiotemporal CNN. However, these techniques are not effective in the future due to continuous improvements in GANs. Style GANs can create fake videos with high accuracy that cannot be easily detected. Hence, deepfake prevention is the need of the hour rather than just mere detection. Methods Recently, blockchain-based ownership methods, image tags, and watermarks in video frames have been used to prevent deepfake. However, this process is not fully functional. An image frame could be faked by copying watermarks and reusing them to create a deepfake. In this research, an enhanced modified version of the steganography technique RivaGAN is used to address the issue. The proposed approach encodes watermarks into features of the video frames by training an “attention model” with the ReLU activation function to achieve a fast learning rate. Results The proposed attention-generating approach has been validated with multiple activation functions and learning rates. It achieved 99.7% accuracy in embedding watermarks into the frames of the video. After generating the attention model, the generative adversarial network has trained using DeepFaceLab 2.0 and has tested the prevention of deepfake attacks using watermark embedded videos comprising 8,074 frames from different benchmark datasets. The proposed approach has acquired a 100% success rate in preventing deepfake attacks. Our code is available at https://github.com/shahidmuneer/deepfakes-watermarking-technique.

show abstract

“…Figure 10 shows a comparison using the DFDC dataset based on accuracy, recall, precision, and F1-score with the existing DL techniques. Additionally, using the FF + + dataset, we evaluated the proposed technique against existing DL models including XceptionNet [46], VGG16 [30], ResNet34 [49], InceptionV3 [44], VGGFace, and E cientNet, and the results are shown in Fig. 10.…”

Section: Comparison With Existing DL Modelsmentioning

confidence: 99%

An Efficient Deepfake Detection Using Robust Deep Learning Approch

Qadir

Mahum

El-Meligy

et al. 2023

Preprint

View full text Add to dashboard Cite

The creation and manipulation of synthetic images have evolved rapidly, creating serious concerns about their effects on society. Although there have been various attempts to identify deep fake videos, these approaches are not universal. Identifying these misleading deepfakes is the first step in preventing them from following on social media sites. We introduce a unique deep-learning technique to identify fraudulent clips. Most deepfake identifiers currently focus on identifying face exchange, lip synchronous, expression modification, puppeteers, and other factors. However, exploring a consistent basis for all forms of fake video and images in real-time forensics is challenging. We propose a hybrid technique that takes input from videos of successive targeted frames, then feeds these to the ResNet-Swish-BiLSTM, an optimized convolutional BiLSTM-based residual network for training and classification. This proposed method helps identify artifacts in deepfake images that do not seem real. To assess the robustness of our proposed model, we used the open deepfake detection challenge dataset (DFDC) and Face Forensics deepfake collections (FF++.) We achieved 96.23% accuracy when using the FF + + digital record. In contrast, we attained 78.33% accuracy using the aggregated records from FF + + and DFDC. We performed extensive experiments and believe that our proposed method provides more significant results than existing techniques.

show abstract

3D CNN Architectures and Attention Mechanisms for Deepfake Detection

Cited by 16 publications

References 48 publications

On the effectiveness of handcrafted features for deepfake video detection

On the effectiveness of handcrafted features for deepfake video detection

Deepfake attack prevention using steganography GANs

An Efficient Deepfake Detection Using Robust Deep Learning Approch

Contact Info

Product

Resources

About