mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation

Abstract

This work presents mEBAL 1 , a multimodal database for eye blink detection and attention level estimation. The eye blink frequency is related to the cognitive activity and automatic detectors of eye blinks have been proposed for many tasks including attention level estimation, analysis of neuro-degenerative diseases, deception recognition, drive fatigue detection, or face anti-spoofing. However, most existing databases and algorithms in this area are limited to experiments involving only a few hundred samples and individual sensors like face cameras. The proposed mEBAL improves previous databases in terms of acquisition sensors and samples. In particular, three different sensors are simultaneously considered: Near Infrared (NIR) and RGB cameras to capture the face gestures and an Electroencephalography (EEG) band to capture the cognitive activity of the user and blinking events. Regarding the size of mEBAL, it comprises 6,000 samples and the corresponding attention level from 38 different students while conducting a number of e-learning tasks of varying difficulty. In addition to presenting mEBAL, we also include preliminary experiments on: i) eye blink detection using Convolutional Neural Networks (CNN) with the facial images, and ii) attention level estimation of the students based on their eye blink frequency.

Face Image Quality Assessment: A Literature Survey

“…Face image quality assessment is an active research area, and can be used for a variety of face recognition related application scenarios, including gender or other soft biometrics recognition [67], attention level estimation [66], emotion analysis [65], etc. The literature surveyed in this work predominantly focused on evaluating their proposed FQA approaches either in terms of predictive performance with respect to given ground truth quality score labels, or in terms of utility [60] [64] for the purpose of aiding face recognition by discarding images based on the assessed quality or some kind of quality-based processing or fusion [82].…”

“…is because this aspect has attracted the predominant interest from researchers so far. Other approaches to FQA such as the prediction of utility in face biometrics tasks like emotion analysis [65], attention level estimation [66], gender or other soft biometrics recognition [67], etc. may open interesting research lines in the future and can take advantage of current developments that employ FQA as FR performance predictor.…”

BlinkLinMulT: Transformer-Based Eye Blink Detection

“…In the work of [6], cropped eyes from RGB texture were used as an input to a VGG16 trained from scratch for binary eye blink state classification. Their main contribution was the introduction of a new blink dataset called mEBAL.…”

“…Then, within the second phase, eye blink detection is conducted via moving-windowed SVD using pixel energy, and eye blink verification is performed via a 2D Pyramidal Bottleneck Block Network (PBBN), which produces the final predictions. They presented improved performance compared to [6,16] by using temporal patterns from a simple energy-based feature.…”

Deepfakes and beyond: A Survey of face manipulation and fake detection

“…Eye blinking [106] has also been studied to detect fake videos. In [92], the authors proposed an algorithm called DeepVision to analyse changes in the blinking patterns.…”