Depression Recognition Using Remote Photoplethysmography From Facial Videos

Abstract

Depression is a mental illness that may be harmful to an individual's health. The detection of mental health disorders in the early stages and a precise diagnosis are critical to avoid social, physiological, or psychological side effects. This work analyzes physiological signals to observe if different depressive states have a noticeable impact on the blood volume pulse (BVP) and the heart rate variability (HRV) response. Although typically, HRV features are calculated from biosignals obtained with contact-based sensors such as wearables, we propose instead a novel scheme that directly extracts them from facial videos, just based on visual information, removing the need for any contact-based device. Our solution is based on a pipeline that is able to extract complete remote photoplethysmography signals (rPPG) in a fully unsupervised manner. We use these rPPG signals to calculate over 60 statistical, geometrical, and physiological features that are further used to train several machine learning regressors to recognize different levels of depression. Experiments on two benchmark datasets indicate that this approach offers comparable results to other audiovisual modalities based on voice or facial expression, potentially complementing them. In addition, the results achieved for the proposed method show promising and solid performance that outperforms hand-engineered methods and is comparable to deep learning-based approaches.

Face2PPG: An Unsupervised Pipeline for Blood Volume Pulse Extraction From Faces

“…Our results are also comparable to some recent supervised methods that require training on videos taken in similar conditions. Additionally, as shown in paper [71], our method is highly efficient, taking only 17 ms per frame, and under 33 ms with face detection and alignment. This outperforms deep learning methods, which generally require longer processing times per frame, underscoring our approach's computational advantage.…”

Enhancing Stress Detection: A Comprehensive Approach through rPPG Analysis and Deep Learning Techniques

“…Similarly, Sabour et al [ 29 ] proposed an rPPG-based stress estimation system with an accuracy of 85.48%. Some other works on the use of rPPG are encouraging, indicating that noncontact measures of some human physiological parameters (e.g., breathing rate (BR) and Heart Rate (HR)) are promising and have great potential for various applications, such as health monitoring [ 47 , 50 ] and affective computing [ 51 , 52 , 53 ]. While these contributions are noteworthy, this paper significantly advances the field by introducing Hybrid Deep Learning (DL) networks and models for rPPG signal reconstruction and Heart Rate (HR) estimation.…”

Depression recognition using voice-based pre-training model

“…Based on visual cues, Guo et al utilized temporal dilated convolutional network (TDCN) for depression detection 8 . Casado et al utilized remote photoplethysmography of facial video to identify depression 9 . Despite the rich information contained in the video, factors like lighting, weather, camera movement, etc., can impact the results.…”

“…In the multi-classification task, the severity of depression was divided into four levels: no depression, mild depression, moderate depression, and severe depression. The PHQ-8 score (range 0-24) was discretized into 4 categories: [0-4], [5][6][7][8][9] , [10][11][12][13][14] and [15][16][17][18][19][20][21][22][23][24], and these four categories are labeled as non, mild, moderate and severe, respectively.…”