Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB

Hernandez-Ortega, Javier; Daza, Roberto; Morales, Aythami; Fiérrez, Julián; Tolosana, Rubén

doi:10.1109/compsac48688.2020.00031

Cited by 11 publications

(17 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work will be oriented to: i) extend ChildCIdb with more participants and acquisition sessions, ii) analyse and improve the accuracy of the children age group detection systems using the remaining tests of ChildCIdb not considered in the present article, iii) study the application of other feature and signal representations of the drawing and screen interaction beyond the ones tested here with special emphasis in recent deep learning methods [41], iv) develop child-independent interaction models for the different test from which child-dependent behaviours can be derived [42], v) correlate the interaction information with the meta-data stored in the dataset like learning outcomes and ADHD [43], vi) combine the information provided by the multiple tests using information fusion methods [44], vii) exploit ChildCIdb in other research problems around e-Learning [16] and e-Health [15], [45], and viii) compare the insights achieved in ChildCI with previous studies focused on the traditional children cognitive development based on Piaget's theory [6].…”

Section: Discussionmentioning

confidence: 99%

“…In this article we present our framework named ChildCI, which is mainly focused on the understanding of CCI with applications to e-Health [15] and e-Learning [16], among others. In particular, the present study introduces all the details regarding the design and development of a new child mobile application, the specific acquisition protocol considered, and the first capturing session of the ChildCI dataset (ChildCIdb).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Child-Computer Interaction with Mobile Devices: Recent Works, New Dataset, and Age Detection

Tolosana,

Ruiz-Garcia,

Vera-Rodriguez

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We overview recent research in Child-Computer Interaction and describe our framework ChildCI intended for: i) generating a better understanding of the cognitive and neuromotor development of children while interacting with mobile devices, and ii) enabling new applications in e-learning and ehealth, among others. Our framework includes a new mobile application, specific data acquisition protocols, and a first release of the ChildCI dataset (ChildCIdb v1), which is planned to be extended yearly to enable longitudinal studies. In our framework children interact with a tablet device, using both a pen stylus and the finger, performing different tasks that require different levels of neuromotor and cognitive skills. ChildCIdb comprises more than 400 children from 18 months to 8 years old, considering therefore the first three development stages of the Piaget's theory. In addition, and as a demonstration of the potential of the ChildCI framework, we include experimental results for one of the many applications enabled by ChildCIdb: children age detection based on device interaction. Different machine learning approaches are evaluated, proposing a new set of 34 global features to automatically detect age groups, achieving accuracy results over 90% and interesting findings in terms of the type of features more useful for this task.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Child-Computer Interaction with Mobile Devices: Recent Works, New Dataset, and Age Detection

Tolosana,

Ruiz-Garcia,

Vera-Rodriguez

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Even though our work has been developed with e-learning in mind [14,15], the contributed resources and methods for eye blink detection can be very useful for other problems as well, e.g. : driver fatigue detection [18], lie detection, DeepFakes detection [24], face anti-spoofing [17], human-computer interfaces [1], and others.…”

Section: Discussionmentioning

confidence: 99%

“…E-learning platforms allow to capture student information to create personalized environments whose contents and methodologies can be adapted dynamically to the different needs of each student. Information such as the performance on questions, the time necessary to perform the different tasks, the emotional state [37], or the the heart rate [16] can be used to understand the student behavior and conditions [15].…”

Section: Introductionmentioning

confidence: 99%

mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation

Daza¹,

Morales²,

Fiérrez³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This work presents mEBAL 1 , a multimodal database for eye blink detection and attention level estimation. The eye blink frequency is related to the cognitive activity and automatic detectors of eye blinks have been proposed for many tasks including attention level estimation, analysis of neuro-degenerative diseases, deception recognition, drive fatigue detection, or face anti-spoofing. However, most existing databases and algorithms in this area are limited to experiments involving only a few hundred samples and individual sensors like face cameras. The proposed mEBAL improves previous databases in terms of acquisition sensors and samples. In particular, three different sensors are simultaneously considered: Near Infrared (NIR) and RGB cameras to capture the face gestures and an Electroencephalography (EEG) band to capture the cognitive activity of the user and blinking events. Regarding the size of mEBAL, it comprises 6,000 samples and the corresponding attention level from 38 different students while conducting a number of e-learning tasks of varying difficulty. In addition to presenting mEBAL, we also include preliminary experiments on: i) eye blink detection using Convolutional Neural Networks (CNN) with the facial images, and ii) attention level estimation of the students based on their eye blink frequency.

show abstract

“…Physiological measurement has provided very valuable information to many different tasks such as e-learning [17], health care [31], human-computer interaction [44], and security [29], among many other tasks.…”

Section: Introductionmentioning

confidence: 99%

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Hernandez-Ortega

Tolosana

Fiérrez

et al. 2022

Advances in Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-Phys.

show abstract

Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB

Cited by 11 publications

References 16 publications

Child-Computer Interaction with Mobile Devices: Recent Works, New Dataset, and Age Detection

Child-Computer Interaction with Mobile Devices: Recent Works, New Dataset, and Age Detection

mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

Contact Info

Product

Resources

About