How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals

Ciftci, Umur Aybars; Demir, İlke; Liu, Yin

doi:10.1109/ijcb48548.2020.9304909

Cited by 70 publications

(31 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A patch-based classifier was introduced in [104] to focus on local patches rather than the global structure. In [105]- [106], the authors extracted features using improved VGG networks. A hypothesis test was performed in [107].…”

Section: ) Deep Learning Based Methodsmentioning

confidence: 99%

Deepfake Detection: A Systematic Literature Review

et al. 2022

View full text Add to dashboard Cite

Over the last few decades, rapid progress in AI, machine learning, and deep learning has resulted in new techniques and various tools for manipulating multimedia. Though the technology has been mostly used in legitimate applications such as for entertainment and education, etc., malicious users have also exploited them for unlawful or nefarious purposes. For example, high-quality and realistic fake videos, images, or audios have been created to spread misinformation and propaganda, foment political discord and hate, or even harass and blackmail people. The manipulated, high-quality and realistic videos have become known recently as Deepfake. Various approaches have since been described in the literature to deal with the problems raised by Deepfake. To provide an updated overview of the research works in Deepfake detection, we conduct a systematic literature review (SLR) in this paper, summarizing 112 relevant articles from 2018 to 2020 that presented a variety of methodologies. We analyze them by grouping them into four different categories: deep learning-based techniques, classical machine learning-based methods, statistical techniques, and blockchain-based techniques. We also evaluate the performance of the detection capability of the various methods with respect to different datasets and conclude that the deep learning-based methods outperform other methods in Deepfake detection.

show abstract

Section: ) Deep Learning Based Methodsmentioning

confidence: 99%

Deepfake Detection: A Systematic Literature Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Ciftci et al [7] first proposed to use physiological signals to detect DeepFakes, their statistics found that the timefrequency characteristics of rPPG signals extracted from synthetic videos are significantly different from those in real videos and they made a series of hand-craft descriptors to identify DeepFakes. In their follow-up work [8], they further used CNN to extract the differencs in rPPG signal and its spectrum, which improved the accuracy of detection.…”

Section: Deepfakes Detection Methodsmentioning

confidence: 99%

“…We selected Xception [6], Inception V3 [39], MesoNet [1] and PPG cell [8] as our baseline for comparison. These methods have well performance in the DeepFakes detection task.…”

Section: Selection Of Baselinementioning

confidence: 99%

“…artifacts generated during affine transformation), which will limit the applicable conditions and reduce the crossdomain robustness of the model. Later, Ciftci et al in their work [7,8] proposed to use remote Photoplethysmography (rPPG), a remotely monitored physiological signal, as a feature for detecting DeepFakes. They proved that in the generated contents, the spatial coherence and temporal consistency of physiological signal will be destroyed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations

Sun¹,

Zhang²,

Qiu³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the rapid development of generation model, AIbased face manipulation technology, which called Deep-Fakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish between real videos and synthetic videos from the variations of facial pixels, which are highly related to physiological information. Our approach first applies Eulerian Video Magnification (EVM) at multiple Gaussian scales to the original video to enlarge the physiological variations caused by the change of facial blood volume, and then transform the original video and magnified videos into a Multi-Scale Eulerian Magnified Spatial-Temporal map (MEMSTmap), which can represent time-varying physiological enhancement sequences on different octaves. Then, these maps are reshaped into frame patches in column units and sent to the vision Transformer to learn the spatio-time descriptors of frame levels. Finally, we sort out the feature embedding and output the probability of judging whether the video is real or fake. We validate our method on the FaceForensics++ and DeepFake Detection datasets. The results show that our model achieves excellent performance in forgery detection, and also show outstanding generalization capability in cross-data domain.

show abstract

“…Having defined our feature space, we explore and analyze the behavior of the predefined features on real and fake gazes. As different generative models create different levels of realism for synthetic faces, they also leave different traces behind [Ciftci et al 2020b]. We experiment with several generator outputs [Dee [n.d.]; Fac [n.d.]b; Thies et al , 2016 corresponding to a real sample.…”

Section: Gaze Authenticitymentioning

confidence: 99%