A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

Hernandez-Ortega, Javier; Fiérrez, Julián; Morales, Aythami; Díaz, David

doi:10.1109/compsac48688.2020.00-53

Cited by 21 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the original DeepPhys model from [6] is not publicly available, instead of training a new CAN from scratch, we decided to initialize DeepFakesON-Phys with the weights from the model pre-trained for heart rate estimation presented in [18], which is also an adaptation of DeepPhys but trained using the COHFACE database [22]. This model also showed to have high accuracy in the heart rate estimation task using real face videos, so our idea is to take benefit of that acquired knowledge to better train DeepFakesON-Phys through a proper fine-tuning process.…”

Section: Deepfakeson-physmentioning

confidence: 99%

“…Observing the results, it seems clear that the fake detectors have learnt to distinguish the spatio-temporal differences between the real/fake faces of Celeb-DF v2 and DFDC Preview databases. Since all the convolutional layers of the proposed fake detector are frozen (the network was originally initialized with the weights from the model trained to predict the heart rate [18]), and we only train the last fully connected layers, we can conclude that the proposed detection approach based on physiological measurement is successful using pulse-related features for distinguishing between real and fake faces. These results prove that the current face manipulation techniques do not pay attention to the heart-rate-related physiological information of the human being when synthesizing fake videos.…”

Section: Deepfakes Detection At Frame Levelmentioning

confidence: 99%

See 1 more Smart Citation

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Hernandez-Ortega

Tolosana

Fiérrez

et al. 2022

Advances in Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-Phys.

show abstract

Section: Deepfakeson-physmentioning

confidence: 99%

Section: Deepfakes Detection At Frame Levelmentioning

confidence: 99%

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Hernandez-Ortega

Tolosana

Fiérrez

et al. 2022

Advances in Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

show abstract

“…These techniques have the target of distinguishing presentation attacks from genuine access attempts based on the analysis of motion. The analysis may consist in detecting any physiological sign of life, for example: pulse [42], eye-blinking [19], facial expression changes [68], or mouth movements. This objective is achieved using knowledge of the human anatomy and physiology.…”

Section: Dynamic Analysismentioning

confidence: 99%

Introduction to Presentation Attack Detection in Face Biometrics and Recent Advances

Hernandez-Ortega¹,

Fiérrez²,

Morales³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The main scope of this chapter is to serve as an introduction to face presentation attack detection, including key resources and advances in the field in the last few years. The next pages present the different presentation attacks that a face recognition system can confront, in which an attacker presents to the sensor, mainly a camera, a Presentation Attack Instrument (PAI), that is generally a photograph, a video, or a mask, to try to impersonate a genuine user.First, we make an introduction of the current status of face recognition, its level of deployment, and its challenges. In addition, we present the vulnerabilities and the possible attacks that a face recognition system may be exposed to, showing that way the high importance of presentation attack detection methods.We review different types of presentation attack methods, from simpler to more complex ones, and in which cases they could be effective. Then, we summarize the most popular presentation attack detection methods to deal with these attacks.Finally, we introduce public datasets used by the research community for exploring vulnerabilities of face biometrics to presentation attacks and developing effective countermeasures against known PAIs.

show abstract

“…One of the most important aspects is the absence of a direct contact between teachers and students. As a result, recent elearning platforms (Hernandez-Ortega et al 2020a) allow to incorporate novel technologies to estimate different factors such as the attention level (Daza et al 2020), the heart rate (Hernandez-Ortega et al 2020b), the emotional state (Shen, Wang, and Shen 2009), and the gaze and head pose (Asteriadis et al 2009).…”

Section: Introductionmentioning

confidence: 99%

ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning

Daza¹,

DeAlcala²,

Morales³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This work presents a feasibility study of remote attention level estimation based on eye blink frequency. We first propose an eye blink detection system based on Convolutional Neural Networks (CNNs), very competitive with respect to related works. Using this detector, we experimentally evaluate the relationship between the eye blink rate and the attention level of students captured during online sessions. The experimental framework is carried out using a public multimodal database for eye blink detection and attention level estimation called mEBAL, which comprises data from 38 students and multiples acquisition sensors, in particular, i) an electroencephalogram (EEG) band which provides the time signals coming from the student's cognitive information, and ii) RGB and NIR cameras to capture the students face gestures. The results achieved suggest an inverse correlation between the eye blink frequency and the attention level. This relation is used in our proposed method called ALEBk for estimating the attention level as the inverse of the eye blink frequency. Our results open a new research line to introduce this technology for attention level estimation on future e-learning platforms, among other applications of this kind of behavioral biometrics based on face analysis.

show abstract

A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

Abstract: https://repositorio.uam.es Esta es la versión de autor del artículo publicado en: This is an author produced version of a paper published in:

Cited by 21 publications

References 12 publications

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Introduction to Presentation Attack Detection in Face Biometrics and Recent Advances

ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning

Contact Info

Product

Resources

About