DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Abstract

This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-Phys.

DeepFake on Face and Expression Swap: A Review

“…it still lacks satisfactory generalization performance with unseen datasets. Another study DeepfakeON-Phys [146], was designed to estimate heart rate from facial video sequences. This deep learning-based model captures spatio-temporal information by analyzing color changes in faces caused by variations in oxygen concentration in the blood.…”

Deepfakes in Cyber Warfare: Threats, Detection, Techniques and Countermeasures

“…Because the creation of Deepfakes can be done with malicious intent, the detection of these fakes presents a potential threat to system security". "According to the findings of the study, machine learning and deep learning models such as CNN and its variations, SVM, LR, and RF and their variants are quite helpful in discriminating between real and fraudulent information that is presented in the form of photographs and videos" [33,34]. Xiaojun Li and his colleagues have developed "a CNN-based model that is capable of effectively recognizing videos that are dishonest by taking into consideration three distinct categories of characteristics.…”

PAD-Phys: Exploiting Physiology for Presentation Attack Detection in Face Biometrics

“…• Experiment: we evaluate presentation attack detection performance for different bona-fide and biometric presentation attack videos using the DeepPhys model [32]. DeepFakes Domain (Level 2).…”