Architectural Tricks for Deep Learning in Remote Photoplethysmography

Abstract

Architectural improvements are studied for convolutional network performing estimation of heart rate (HR) values on color signal patches. Color signals are time series of color components averaged over facial regions recorded by webcams in two scenarios: Stationary (without motion of a person) and Mixed Motion (different motion patterns of a person). HR estimation problem is addressed as a classification task, where classes correspond to different heart rate values within the admissible range of [40; 125] bpm. Both adding convolutional filtering layers after fully connected layers and involving combined loss function where first component is a cross entropy and second is a squared error between the network output and smoothed one-hot vector, lead to better performance of HR estimation model in Stationary and Mixed Motion scenarios.

Machine learning models and facial regions videos for estimating heart rate: a review on Patents, Datasets and Literature

“…[ 44,10,42,11,43,12,40,38,6] reduced the ROIs of the face using facial landmark detectors, while [36,39,45,9,18,37] utilized the full face as model input. In [44,10], they performed removal of irrelevant signals such as tremor, illumination changes, and so on in order to locate and cut out ROIs to estimate heart rate.…”

“…When there is ROI clipping, [42] pointed out that, despite requiring less processing in terms of treating irrelevant regions, the challenge of providing a consistent detection of face landmarks to smooth noisy signals occurred. [44,10,36,42,39,45,11,9,43,18,12,37,40,38,6] estimated heart rate using face videos, while [9,46,5,1,41] estimated additional physiological signals using contact devices or heart rate estimation techniques.…”

“…Furthermore, [11,43,38] employed other techniques to obtain heart rate. For face detection of ROIs, [18,40] employed Opencv, while [44,10] used a Single Shot Multibox Detector (SSD) network. ROIs are utilized to monitor color changes at places where heart rate estimate is achieved by viewing such changes.…”

Biometric Signals Estimation Using Single Photon Camera and Deep Learning

“…Some surveys on the state of the art of this field could be found in [1,[18][19][20]. While machine learning techniques are widely used in contact PPG applications [21], recent works [22][23][24] explored the opportunity of also using deep learning methods in remote PPG applications. All these works completely substitute the classical signal processing techniques with deep learning ones using an end-to-end network, as in [22,24], or using two consecutive neural networks, as in [23].…”

“…While machine learning techniques are widely used in contact PPG applications [21], recent works [22][23][24] explored the opportunity of also using deep learning methods in remote PPG applications. All these works completely substitute the classical signal processing techniques with deep learning ones using an end-to-end network, as in [22,24], or using two consecutive neural networks, as in [23]. On one hand, the use of an end-to-end deep learning model has proved to achieve state-of-the-art results on many computer vision tasks such as image segmentation, object detection, and many others.…”