Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics

Zhu, Yu; Shang, Yuanyuan; Shao, Zhuhong; Guo, Guodong

doi:10.1109/taffc.2017.2650899

Cited by 174 publications

(154 citation statements)

References 30 publications

Supporting

Mentioning

140

Contrasting

Unclassified

Order By: Relevance

“…Egyptians believed that the human face proportions are closely linked to consciousness and how feelings are expressed, while in Chinese culture, the facial structure played a major role in Daoist philosophy and was thought to reveal information about the mental and physical state of an individual [1]. Although this practice was disputed throughout the Middle Ages and up until the nineteenth century, it has regained interest in the latest years, and several recent studies showed that facial appearance is indeed linked to different psychological processes and behaviors [2][3][4]. Recent research showed that people's evaluation of others is also closely related to their physical appearance, as we tend to interact with other people based on our first impression [5], and this first impression is in many ways influenced by the appearance of the people we interact with [6].…”

Section: Introductionmentioning

confidence: 99%

“…Although at first focused on predicting the emotional state of people [12,13], as Facial Expression Recognition (FER) systems gained momentum and started achieving acceptable prediction accuracy, recent research papers have begun using facial features analysis for more complex tasks, such as tracking and predicting eye gaze [14,15], predicting driver attention for car accident prevention [14,16], predicting stress levels [2,17], diagnosing depression [3], assessing the facial attractiveness of individuals [18], evaluating people's trust [19], and predicting personality traits [4,[20][21][22][23]. All these research studies showed that the face indeed conveys information that can be analyzed to predict different psychological features of an individual.…”

Section: Introductionmentioning

confidence: 99%

“…It could be useful in applications for career development and counseling in the human resources or academic areas [30,31], adaptive e-learning systems [32], diagnosis of mental health disorders (borderline personality disorder [33], depression [3], schizophrenia [34], eating disorder [35] or sleep disorders [36]), virtual psychologist applications [37], and personalized health assistance [38]. It was also shown that there are links between common physical diseases (such as heart attacks, diabetes, cancer, strokes, arthritis, hypertension, and respiratory disease) and Big Five personality traits [39] such that these diseases influence the age-related personality accelerating with 2.5 years decrease for extraversion, 5 years decrease for conscientiousness, 1.6 years decrease for openness, and 1.9 years increase for emotional stability.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Gavrilescu

Vizireanu

2017

J Image Video Proc.

View full text Add to dashboard Cite

We propose a novel three-layered neural network-based architecture for predicting the Sixteen Personality Factors from facial features analyzed using Facial Action Coding System. The proposed architecture is built on three layers: a base layer where the facial features are extracted from each video frame using a multi-state face model and the intensity levels of 27 Action Units (AUs) are computed, an intermediary level where an AU activity map is built containing all AUs' intensity levels fetched from the base layer in a frame-by-frame manner, and a top layer consisting of 16 feed-forward neural networks trained via backpropagation which analyze the patterns in the AU activity map and compute scores from 1 to 10, predicting each of the 16 personality traits. We show that the proposed architecture predicts with an accuracy of over 80%: warmth, emotional stability, liveliness, social boldness, sensitivity, vigilance, and tension. We also show there is a significant relationship between the emotions elicited to the analyzed subjects and high prediction accuracy obtained for each of the 16 personality traits as well as notable correlations between distinct sets of AUs present at high-intensity levels and increased personality trait prediction accuracy. The system converges to a stable result in no more than 1 min, making it faster and more practical than the Sixteen Personality Factors Questionnaire and suitable for real-time monitoring of people's personality traits.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Gavrilescu

Vizireanu

2017

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…In our model, N parallel convolutional layers take xt as input of raw speech and create N di erent sequences of feature maps by convoluting xt with a set of lters of di erent lengths. e output of convoluting each layer consisting of nw lters having widths kw and strides dw are computed using (1).…”

Section: Feature Extraction Blockmentioning

confidence: 99%

Direct Modelling of Speech Emotion from Raw Speech

et al. 2019

View full text Add to dashboard Cite

Speech emotion recognition is a challenging task and heavily depends on hand-engineered acoustic features, which are typically cra ed to echo human perception of speech signals. However, a lter bank that is designed from perceptual evidence is not always guaranteed to be the best in a statistical modelling framework where the end goal is for example emotion classi cation. is has fuelled the emerging trend of learning representations from raw speech especially using deep learning neural networks. In particular, a combination of Convolution Neural Networks (CNNs) and Long Short Term Memory (LSTM) have gained great traction for the intrinsic property of LSTM in learning contextual information crucial for emotion recognition; and CNNs been used for its ability to overcome the scalability problem of regular neural networks. In this paper, we show that there are still opportunities to improve the performance of emotion recognition from the raw speech by exploiting the properties of CNN in modelling contextual information. We propose the use of parallel convolutional layers to harness multiple temporal resolutions in the feature extraction block that is jointly trained with the LSTM based classication network for the emotion recognition task. Our results suggest that the proposed model can reach the performance of CNN trained with hand-engineered features from both IEMO-CAP and MSP-IMPROV datasets.

show abstract

“…Dibeklioglu et al [32] employed Stacked Denoising Autoencoders in a multimodal context to perform video classification according to three levels of depressive symptomatology on the Pittsburgh dataset. Moreover, Zhu et al [33] employed Deep Convolutional Neural Networks to achieve the highest performance among the unimodal (visual) approaches addressing the aim of AVEC'13 and AVEC'14 competitions.…”

Section: Deep Learningmentioning

confidence: 99%

Quantitative comparison of motion history image variants for video-based depression assessment

Pampouchidou

Pediaditis

Maridaki

et al. 2017

J Image Video Proc.

View full text Add to dashboard Cite

Depression is the most prevalent mood disorder and a leading cause of disability worldwide. Automated video-based analyses may afford objective measures to support clinical judgments. In the present paper, categorical depression assessment is addressed by proposing a novel variant of the Motion History Image (MHI) which considers Gabor-inhibited filtered data instead of the original image. Classification results obtained with this method on the AVEC'14 dataset are compared to those derived using (a) an earlier MHI variant, the Landmark Motion History Image (LMHI), and (b) the original MHI. The different motion representations were tested in several combinations of appearance-based descriptors, as well as with the use of convolutional neural networks. The F1 score of 87.4% achieved in the proposed work outperformed previously reported approaches.

show abstract

Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics

Cited by 174 publications

References 30 publications

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Direct Modelling of Speech Emotion from Raw Speech

Quantitative comparison of motion history image variants for video-based depression assessment

Contact Info

Product

Resources

About