Two-Branch Recurrent Network for Isolating Deepfakes in Videos

Masi, Iacopo; Killekar, Aditya; Mascarenhas, Royston Marian; Gurudatt, Shenoy Pratik; AbdAlmageed, Wael

doi:10.1007/978-3-030-58571-6_39

Cited by 215 publications

(100 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In De Lima et al [ 41 ], the face regions are detected using the RetinaFace, and then the 3D CNNs are applied to learn the spatial-temporal features and detect the fake in the Celeb-DF videos dataset. In Masi et al [ 42 ], the face frames are aligned, and then two DenseBlocks models are applied to merge the information from the frequency and color domains. These blocks are followed by Bi-LSTM to learn the temporal information and detect the deepfake videos.…”

Section: Literature Reviewmentioning

confidence: 99%

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

Ismail

Elpeltagy

Zaki

et al. 2021

Sensors

View full text Add to dashboard Cite

Currently, face-swapping deepfake techniques are widely spread, generating a significant number of highly realistic fake videos that threaten the privacy of people and countries. Due to their devastating impacts on the world, distinguishing between real and deepfake videos has become a fundamental issue. This paper presents a new deepfake detection method: you only look once–convolutional neural network–extreme gradient boosting (YOLO-CNN-XGBoost). The YOLO face detector is employed to extract the face area from video frames, while the InceptionResNetV2 CNN is utilized to extract features from these faces. These features are fed into the XGBoost that works as a recognizer on the top level of the CNN network. The proposed method achieves 90.62% of an area under the receiver operating characteristic curve (AUC), 90.73% accuracy, 93.53% specificity, 85.39% sensitivity, 85.39% recall, 87.36% precision, and 86.36% F1-measure on the CelebDF-FaceForencics++ (c23) merged dataset. The experimental study confirms the superiority of the presented method as compared to the state-of-the-art methods.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

Ismail

Elpeltagy

Zaki

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Two-stream [68] 53.8 Meso4 [33] 54.8 HeadPose [61] 54.6 FWA [11] 56.9 VA-MLP [45] 55.0 Xception-c40 [58] 65.5 Multi-task [47] 54.3 Capsule [69] 57.5 TBRN [70] 73.41 Face X-ray [71] 80.58 PPA [72] 83.10 FakeCatcher [44] 91.50…”

Section: Models Auc (%)mentioning

confidence: 99%

ADD: Attention-Based DeepFake Detection Approach

Khormali

Yuan

2021

BDCC

View full text Add to dashboard Cite

Recent advancements of Generative Adversarial Networks (GANs) pose emerging yet serious privacy risks threatening digital media’s integrity and trustworthiness, specifically digital video, through synthesizing hyper-realistic images and videos, i.e., DeepFakes. The need for ascertaining the trustworthiness of digital media calls for automatic yet accurate DeepFake detection algorithms. This paper presents an attention-based DeepFake detection (ADD) method that exploits the fine-grained and spatial locality attributes of artificially synthesized videos for enhanced detection. ADD framework is composed of two main components including face close-up and face shut-off data augmentation methods and is applicable to any classifier based on convolutional neural network architecture. ADD first locates potentially manipulated areas of the input image to extract representative features. Second, the detection model is forced to pay more attention to these forgery regions in the decision-making process through a particular focus on interpreting the sample in the learning phase. ADD’s performance is evaluated against two challenging datasets of DeepFake forensics, i.e., Celeb-DF (V2) and WildDeepFake. We demonstrated the generalization of ADD by evaluating four popular classifiers, namely VGGNet, ResNet, Xception, and MobileNet. The obtained results demonstrate that ADD can boost the detection performance of all four baseline classifiers significantly on both benchmark datasets. Particularly, ADD with ResNet backbone detects DeepFakes with more than 98.3% on Celeb-DF (V2), outperforming state-of-the-art DeepFake detection methods.

show abstract

“…In [ 167 ], authors designed a two-branch CNN to exploit the distribution differences between pixels in the face region and the background. Masi et al [ 168 ] proposed a two-branch LSTM to combine color and frequency information. A multi-scale Laplacian-of-Gaussian operator was used in their method, which acted as a band-pass filter to amplify the artifacts.…”

Section: Other Specific Forensic Problemsmentioning

confidence: 99%

“…We can see that most methods achieved a good performance on a binary classification for GAN-generated images. In the case of videos as input (cf., column of “Video”) and using FaceForensics++ as dataset, the use of an architecture that can track changes among frames, such as LSTM in the method of [ 168 ], leads to very good performance. On images (cf., column of “Image”), results from [ 166 ] show that traces of current GANs are easy to detect.…”

Section: Other Specific Forensic Problemsmentioning

confidence: 99%

A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics

Camacho

Wang

2021

J. Imaging

View full text Add to dashboard Cite

Seeing is not believing anymore. Different techniques have brought to our fingertips the ability to modify an image. As the difficulty of using such techniques decreases, lowering the necessity of specialized knowledge has been the focus for companies who create and sell these tools. Furthermore, image forgeries are presently so realistic that it becomes difficult for the naked eye to differentiate between fake and real media. This can bring different problems, from misleading public opinion to the usage of doctored proof in court. For these reasons, it is important to have tools that can help us discern the truth. This paper presents a comprehensive literature review of the image forensics techniques with a special focus on deep-learning-based methods. In this review, we cover a broad range of image forensics problems including the detection of routine image manipulations, detection of intentional image falsifications, camera identification, classification of computer graphics images and detection of emerging Deepfake images. With this review it can be observed that even if image forgeries are becoming easy to create, there are several options to detect each kind of them. A review of different image databases and an overview of anti-forensic methods are also presented. Finally, we suggest some future working directions that the research community could consider to tackle in a more effective way the spread of doctored images.

show abstract

Two-Branch Recurrent Network for Isolating Deepfakes in Videos

Cited by 215 publications

References 39 publications

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

ADD: Attention-Based DeepFake Detection Approach

A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics

Contact Info

Product

Resources

About