FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

Ciftci, Umur Aybars; Demir, İlke; Liu, Yin

doi:10.1109/tpami.2020.3009287

Cited by 275 publications

(167 citation statements)

References 77 publications

Supporting

Mentioning

164

Contrasting

Unclassified

Order By: Relevance

“…erefore, the RCNN-based methods can make full use of spatial and temporal information of Deepfake videos. Moreover, some Deepfake detection methods [12,25] are based on traditional machine learning methods, Yang et al [12] and Ciftci et al [25] used SVM (support vector machine) as a classifier by extracting handcrafted features, such as biological signals. Finally, the methods mentioned above are summarized in Table 1.…”

Section: Deepfake Detectionmentioning

confidence: 99%

“…In addition, the loss of label classifier G y is also minimized. e overall loss function of DANN can be formalized as [20] CNN Multitask ForensicTransfer [21] CNN Multitask Face X-Ray [22] CNN Multitask Qian et al [24] CNN Frequency Li et al [11] CNN + LSTM Handcrafted Guera et al [17] CNN + LSTM RGB Chen et al [18] CNN + LSTM RGB Yang et al [12] SVM Handcrafted FakeCatcher [25] SVM Handcrafted…”

Section: Domain-adversarial Networkmentioning

confidence: 99%

See 1 more Smart Citation

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

Chen

Tan

2021

Security and Communication Networks

View full text Add to dashboard Cite

Recently, various Deepfake detection methods have been proposed, and most of them are based on convolutional neural networks (CNNs). These detection methods suffer from overfitting on the source dataset and do not perform well on cross-domain datasets which have different distributions from the source dataset. To address these limitations, a new method named FeatureTransfer is proposed in this paper, which is a two-stage Deepfake detection method combining with transfer learning. Firstly, The CNN model pretrained on a third-party large-scale Deepfake dataset can be used to extract the more transferable feature vectors of Deepfake videos in the source and target domains. Secondly, these feature vectors are fed into the domain-adversarial neural network based on backpropagation (BP-DANN) for unsupervised domain adaptive training, where the videos in the source domain have real or fake labels, while the videos in the target domain are unlabelled. The experimental results indicate that the proposed method FeatureTransfer can effectively solve the overfitting problem in Deepfake detection and greatly improve the performance of cross-dataset evaluation.

show abstract

Section: Deepfake Detectionmentioning

confidence: 99%

Section: Domain-adversarial Networkmentioning

confidence: 99%

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

Chen

Tan

2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Since deepfake videos circulated in social media have brought serious concerns such as through celebrity pornographic videos, fake news, hoaxes, and financial fraud, which largely impairs the integrity of social media, deepfake detection has attracted a lot of attention in the recent computer vision research. In terms of the roles of rPPG for deepfake detection, FakeCatcher [19] explored the discriminative features of rPPG signals extracted from facial videos and utilized them for deepfake detection.…”

Section: B Biometric Privacy Protectionmentioning

confidence: 99%

“…Since PulseEdit can edit rPPG signals in videos, we expect that PulseEdit, as an adversarial operation, can circumvent rPPG-based liveness detection [16], [18] and rPPG-based deepfake detection [19]. Thus, we conducted experiments on the HKBUMARsV1+ dataset [17] for liveness detection and the Celeb-DFv1 dataset [43] for deepfake detection to evaluate the effectiveness of PulseEdit on above two aspects, respectively.…”

Section: Analysis Of Adversarial Scenariosmentioning

confidence: 99%

PulseEdit: Editing Physiological Signal in Facial Videos for Privacy Protection

Chen¹,

Liao²,

Wu³

2021

Preprint

View full text Add to dashboard Cite

Recent studies have shown that physiological signals can be remotely captured from human faces using a portable color camera under ambient light. This technology, namely remote photoplethysmography (rPPG), can be used to collect users' physiological status who are sitting in front of a camera, which may raise physiological privacy issues. To avoid the privacy abuse of the rPPG technology, this paper develops PulseEdit, a novel and efficient algorithm that can edit the physiological signals in facial videos without affecting visual appearance to protect the user's physiological signal from disclosure. PulseEdit can either remove the trace of the physiological signal in a video or transform the video to contain a target physiological signal chosen by a user. Experimental results show that PulseEdit can effectively edit physiological signals in facial videos and prevent heart rate measurement based on rPPG. It is possible to utilize PulseEdit in adversarial scenarios against some rPPG-based visual security algorithms. We present analyses on the performance of PulseEdit against rPPG-based liveness detection and rPPG-based deepfake detection, and demonstrate its ability to circumvent these visual security algorithms.

show abstract

“…The eye-blinking datasets is the first available dataset which specially designed for the eye-blinking detection. The experiment's results demonstrate the efficacy of the suggested approach in detecting false images.Other biological signals such as heartbeat have been shown to be a reliable predictor for real video.Prior research has shown that, in addition to biological signals, there is a close relationship between various audio-visual modalities of the same sample[36] [37][38] [39][40]. Mittal et al…”

mentioning

confidence: 99%

Deepfakes Detection Techniques Using Deep Learning: A Survey

Almars¹

2021

JCC

View full text Add to dashboard Cite

Deep learning is an effective and useful technique that has been widely applied in a variety of fields, including computer vision, machine vision, and natural language processing. Deepfakes uses deep learning technology to manipulate images and videos of a person that humans cannot differentiate them from the real one. In recent years, many studies have been conducted to understand how deepfakes work and many approaches based on deep learning have been introduced to detect deepfakes videos or images. In this paper, we conduct a comprehensive review of deepfakes creation and detection technologies using deep learning approaches. In addition, we give a thorough analysis of various technologies and their application in deepfakes detection. Our study will be beneficial for researchers in this field as it will cover the recent state-of-art methods that discover deepfakes videos or images in social contents. In addition, it will help comparison with the existing works because of the detailed description of the latest methods and dataset used in this domain.

show abstract

FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

Cited by 275 publications

References 77 publications

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

PulseEdit: Editing Physiological Signal in Facial Videos for Privacy Protection

Deepfakes Detection Techniques Using Deep Learning: A Survey

Contact Info

Product

Resources

About