Detecting Deepfakes with Metric Learning

Over the last few decades, rapid progress in AI, machine learning, and deep learning has resulted in new techniques and various tools for manipulating multimedia. Though the technology has been mostly used in legitimate applications such as for entertainment and education, etc., malicious users have also exploited them for unlawful or nefarious purposes. For example, high-quality and realistic fake videos, images, or audios have been created to spread misinformation and propaganda, foment political discord and hate, or even harass and blackmail people. The manipulated, high-quality and realistic videos have become known recently as Deepfake. Various approaches have since been described in the literature to deal with the problems raised by Deepfake. To provide an updated overview of the research works in Deepfake detection, we conduct a systematic literature review (SLR) in this paper, summarizing 112 relevant articles from 2018 to 2020 that presented a variety of methodologies. We analyze them by grouping them into four different categories: deep learning-based techniques, classical machine learning-based methods, statistical techniques, and blockchain-based techniques. We also evaluate the performance of the detection capability of the various methods with respect to different datasets and conclude that the deep learning-based methods outperform other methods in Deepfake detection.

show abstract

“…Machine learningbased methods [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [85], [87], [88], [143] 16 18%…”

Section: • Integrates the Critical Features Of Ipfs [114]-basedmentioning

confidence: 99%

Deepfake Detection: A Systematic Literature Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…But this kind of face recognition system without forgery algorithms or modules usually cannot resist face-swap attacks. [26] 0.992 -Face X-ray [27] 0.748 -Fakespotter [28] 0.668 -Mesonet [29] -0.753 Face recognition technology, due to its convenience and remarkable, has been applied in a few interactive intelligent applications. In those scenarios with high security requirements, these easily exposed face recognition systems have a number of security implications.…”

Section: Discussionmentioning

confidence: 99%

Channel-Wise Spatiotemporal Aggregation Technology for Face Video Forensics

Liu

Fei

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

Recent progress in deep learning, in particular the generative models, makes it easier to synthesize sophisticated forged faces in videos, leading to severe threats on social media about personal privacy and reputation. It is therefore highly necessary to develop forensics approaches to distinguish those forged videos from the authentic. Existing works are absorbed in exploring frame-level cues but insufficient in leveraging affluent temporal information. Although some approaches identify forgeries from the perspective of motion inconsistency, there is so far not a promising spatiotemporal feature fusion strategy. Towards this end, we propose the Channel-Wise Spatiotemporal Aggregation (CWSA) module to fuse deep features of continuous video frames without any recurrent units. Our approach starts by cropping the face region with some background remained, which transforms the learning objective from manipulations to the difference between pristine and manipulated pixels. A deep convolutional neural network (CNN) with skip connections that are conducive to the preservation of detection-helpful low-level features is then utilized to extract frame-level features. The CWSA module finally makes the real or fake decision by aggregating deep features of the frame sequence. Evaluation against a list of large facial video manipulation benchmarks has illustrated its effectiveness. On all three datasets, FaceForensics++, Celeb-DF, and DeepFake Detection Challenge Preview, the proposed approach outperforms the state-of-the-art methods with significant advantages.

show abstract

“…Then, these various architectures are trained on different datasets and tested on the Celeb-DF dataset. The work in Kumar et al [ 34 ] extracts the face regions from video frames of the Celeb-DF dataset using the MTCNN and then applies the XceptionNet architecture. The authors in Khalil et al [ 35 ] use YOLO v3 for face detection.…”

Section: Literature Reviewmentioning

confidence: 99%

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

Ismail

Elpeltagy

Zaki

et al. 2021

Sensors

View full text Add to dashboard Cite

Currently, face-swapping deepfake techniques are widely spread, generating a significant number of highly realistic fake videos that threaten the privacy of people and countries. Due to their devastating impacts on the world, distinguishing between real and deepfake videos has become a fundamental issue. This paper presents a new deepfake detection method: you only look once–convolutional neural network–extreme gradient boosting (YOLO-CNN-XGBoost). The YOLO face detector is employed to extract the face area from video frames, while the InceptionResNetV2 CNN is utilized to extract features from these faces. These features are fed into the XGBoost that works as a recognizer on the top level of the CNN network. The proposed method achieves 90.62% of an area under the receiver operating characteristic curve (AUC), 90.73% accuracy, 93.53% specificity, 85.39% sensitivity, 85.39% recall, 87.36% precision, and 86.36% F1-measure on the CelebDF-FaceForencics++ (c23) merged dataset. The experimental study confirms the superiority of the presented method as compared to the state-of-the-art methods.

show abstract

Detecting Deepfakes with Metric Learning

Cited by 58 publications

References 17 publications

Deepfake Detection: A Systematic Literature Review

Deepfake Detection: A Systematic Literature Review

Channel-Wise Spatiotemporal Aggregation Technology for Face Video Forensics

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

Contact Info

Product

Resources

About