Automatic Identification of Depression Using Facial Images with Deep Convolutional Neural Network

Kong, Xinru; Yao, Yan; Wang, Cuiying; Wang, Yuangeng; Teng, Jing; Qi, Xianghua

doi:10.12659/msm.936409

Cited by 7 publications

(11 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extracting facial features to assess mental health and emotions has received significant attention in computer vision, with applications spanning from education to healthcare [51]. Here, many studies have explored facial expressions, gaze patterns, and the overall composition of images to extract visual markers symptomatic of depression [38,43,46]. However, most of these studies are conducted in controlled environments or rely on participants deliberately capturing their images, which could inadvertently influence their emotional portrayal.…”

Section: Smartphone Images In Controlled Settings For Mental Healthmentioning

confidence: 99%

“…However, most of these studies are conducted in controlled environments or rely on participants deliberately capturing their images, which could inadvertently influence their emotional portrayal. For instance, Kong et al [38] captured photographs using a tablet in a standardized clinical setting. Participants were asked to sit before a white background, remove hats or glasses, and tie up long hair to expose their ears; the users looked straight ahead with relaxed expressions as instructed.…”

Section: Smartphone Images In Controlled Settings For Mental Healthmentioning

confidence: 99%

“…Most of the prior research utilizing facial images to detect depression focus on capturing these images in controlled settings, where individuals may be instructed to perform specific actions [22,38,43,46]. These face features are not authentic as they are performative and are influenced by biases such as social desirability and self-presentation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MoodCapture: Depression Detection using In-the-Wild Smartphone Images

Nepal,

Pillai,

Wang

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: "I have felt down, depressed, or hopeless". Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our posthoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing; • Applied computing → Health informatics.

show abstract

Section: Smartphone Images In Controlled Settings For Mental Healthmentioning

confidence: 99%

Section: Smartphone Images In Controlled Settings For Mental Healthmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MoodCapture: Depression Detection using In-the-Wild Smartphone Images

Nepal,

Pillai,

Wang

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…In the second branch, the interaction between w and N dimensions is established by performing the dimension transformation F w → F w ∈ R h×N×w . The calculation process is the same as Equation (10), and the result is F 2 w ∈ R h×N×w . Then, the dimension is restored:…”

Section: Triplet Attentionmentioning

confidence: 99%

“…In recent years, with the development of computer technology and artificial intelligence, more and more studies have begun to explore the use of computer-aided diagnosis methods to recognize depression, such as using computers to recognize facial expressions [ 10 , 11 ], Electroencephalogram (EEG) [ 12 ], Electrocardiograms (ECG) [ 13 ], and speech [ 14 ] to analyze whether the test subject is depressed. The above methods can be roughly divided into three steps: (1) data collection, which uses sensors, cameras, microphones, and other devices to collect physiological data such as facial expressions, EEG, ECG, and speech from test subjects; (2) data processing, which preprocesses and cleans the collected data and performs data transformation and normalization; (3) feature extraction and recognition, which uses machine learning and deep learning algorithms to extract features related to depression from the processed data.…”

Section: Introductionmentioning

confidence: 99%

FBANet: Transfer Learning for Depression Recognition Using a Feature-Enhanced Bi-Level Attention Network

Wang,

Zhang,

Huang

et al. 2023

Entropy

View full text Add to dashboard Cite

The House-Tree-Person (HTP) sketch test is a psychological analysis technique designed to assess the mental health status of test subjects. Nowadays, there are mature methods for the recognition of depression using the HTP sketch test. However, existing works primarily rely on manual analysis of drawing features, which has the drawbacks of strong subjectivity and low automation. Only a small number of works automatically recognize depression using machine learning and deep learning methods, but their complex data preprocessing pipelines and multi-stage computational processes indicate a relatively low level of automation. To overcome the above issues, we present a novel deep learning-based one-stage approach for depression recognition in HTP sketches, which has a simple data preprocessing pipeline and calculation process with a high accuracy rate. In terms of data, we use a hand-drawn HTP sketch dataset, which contains drawings of normal people and patients with depression. In the model aspect, we design a novel network called Feature-Enhanced Bi-Level Attention Network (FBANet), which contains feature enhancement and bi-level attention modules. Due to the limited size of the collected data, transfer learning is employed, where the model is pre-trained on a large-scale sketch dataset and fine-tuned on the HTP sketch dataset. On the HTP sketch dataset, utilizing cross-validation, FBANet achieves a maximum accuracy of 99.07% on the validation dataset, with an average accuracy of 97.71%, outperforming traditional classification models and previous works. In summary, the proposed FBANet, after pre-training, demonstrates superior performance on the HTP sketch dataset and is expected to be a method for the auxiliary diagnosis of depression.

show abstract