Human Behaviour-Based Automatic Depression Analysis Using Hand-Crafted Statistics and Deep Learned Spectral Features

Song, Siyang; Shen, Linlin; Valstar, Michel

doi:10.1109/fg.2018.00032

Cited by 93 publications

(58 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper will focus on techniques for accurate detection of depression levels over time based on facial expression captured in videos. Although spatial information is essential, the dynamics of facial behavior is also very important to interpret depression [10], [11], [12]. Alghowinem et al [13] have identified behavioural cues related to depressed individuals, such as slower head movements and avoidance of eye contact.…”

Section: Introductionmentioning

confidence: 99%

“…Deep learning architectures, and in particular CNNs, provide state-of-the-art performance in many visual recognition applications, such as image classification [14] and object detection [15], as well as assisted medical diagnosis [16]. In depression detection, deep learning architectures that process on videos typically exploit spatial and temporal information separately (e.g., by cascading a 2D CNN and then a recurrent NN), which deteriorate the modeling of spatio-temporal relationships [11], [17]. A deep two-stream architecture has also been proposed to exploit facial appearance and facial optical flow [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

Melo

Granger

Hadid

2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

Deep learning architectures have been successfully applied in video-based health monitoring, to recognize distinctive variations in the facial appearance of subjects. To detect patterns of variation linked to depressive behavior, deep neural networks (NNs) typically exploit spatial and temporal information separately by, e.g., cascading a 2D convolutional NN (CNN) with a recurrent NN (RNN), although the intrinsic spatio-temporal relationships can deteriorate. With the recent advent of 3D CNNs like the convolutional 3D (C3D) network, these spatio-temporal relationships can be modeled to improve performance. However, the accuracy of C3D networks remain an issue when applied to depression detection. In this paper, the fusion of diverse C3D predictions are proposed to improve accuracy, where spatio-temporal features are extracted from global (full-face) and local (eyes) regions of subject. This allows to increasingly focus on a local facial region that is highly relevant for analyzing depression. Additionally, the proposed network integrates 3D Global Average Pooling in order to efficiently summarize spatio-temporal features without using fully-connected layers, and thereby reduce the number of model parameters and potential over-fitting. Experimental results on the Audio Visual Emotion Challenge (AVEC 2013 and AVEC 2014) depression datasets indicates that combining the responses of global and local C3D networks achieves a higher level of accuracy than state-of-the-art systems.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

Melo

Granger

Hadid

2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

show abstract

“…We examined 42 eyes from 42 adults with correctedto-normal vision (median [interquartile range, IQR] age: 26 [22][23][24][25][26][27][28][29] years), and 14 eyes from seven adults with an established diagnosis of glaucoma (69 [64][65][66][67][68][69][70][71][72][73][74] years of age).…”

Section: Participants and Proceduresmentioning

confidence: 99%

The Human Touch: Using a Webcam to Autonomously Monitor Compliance During Visual Field Assessments

Jones

Demaria

Tigchelaar

et al. 2020

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

To explore the feasibility of using various easy-to-obtain biomarkers to monitor non-compliance (measurement error) during visual field assessments. Methods: Forty-two healthy adults (42 eyes) and seven glaucoma patients (14 eyes) underwent two same-day visual field assessments. An ordinary webcam was used to compute seven potential biomarkers of task compliance, based primarily on eye gaze, head pose, and facial expression. We quantified the association between each biomarker and measurement error, as defined by (1) test-retest differences in overall test scores (mean sensitivity), and (2) failures to respond to visible stimuli on individual trials (stimuli −3 dB or more brighter than threshold). Results: In healthy eyes, three of the seven biomarkers were significantly associated with overall (test-retest) measurement error (P = 0.003-0.007), and at least two others exhibited possible trends (P = 0.052-0.060). The weighted linear sum of all seven biomarkers was associated with overall measurement error, in both healthy eyes (r = 0.51, P < 0.001) and patients (r = 0.65, P < 0.001). Five biomarkers were each associated with failures to respond to visible stimuli on individual trials (all P < 0.001). Conclusions: Inexpensive, autonomous measures of task compliance are associated with measurement error in visual field assessments, in terms of both the overall reliability of a test and failures to respond on particular trials ("lapses"). This could be helpful for identifying low-quality assessments and for improving assessment techniques (e.g., by discounting suspect responses or by automatically triggering comfort breaks or encouragement). Translational Relevance: This study explores a potential way of improving the reliability of visual field assessments, a crucial but notoriously unreliable clinical measure.

show abstract

“…To understand which applies, we first need to briefly unpack the discussion around if emotion data is biometric data, because if it is, processing will require the explicit consent of the data subject (Art 9, GDPR). Data concerning health, particularly mental health (art 4(15) could conceivably be read from facial coding (e.g., see work of Song, et al [2018] on detecting depression using CV), and thus require, we argue requires explicit consent.…”

Section: Gdpr: Eai Testing Established Data Protection Principlesmentioning

confidence: 99%

‘This time with feeling?’ Assessing EU data governance implications of out of home appraisal based emotional AI

McStay¹,

Urquhart²

2019

View full text Add to dashboard Cite

The boundaries of personal space and borders of bodily integrity are being tested by deployments of emotional artificial intelligence (EAI) in private and public spaces. By means of sensing, seeing and machine learning of facial expressions, voice, gaze, gestures and range of physiological signals (heart rate, skin conductivity and temperature, muscle activity, body temperature, respiration and other bio-signals), the goal is to make interior emotional life machine-readable for personal, commercial and security objectives.In this paper, we focus on computer vision and face-based analytics to consider the nature, method and development of facial coding, the potential demise of existing approaches, and the rise of even more invasive methods. Criticisms of facial coding have long existed, but recent scholarship and industrial development signals a lack of confidence in ‘basic emotions’ and a turn to appraisal-based accounts of emotion. This inevitably entails use of data about internal physiological and experiential contexts, but also factors external to an individual. To explore this, this paper asks and answers the following question: With regard to deployment in out-of-home situations, what are the legal and privacy implications of appraisal-based emotion capture?

show abstract

Human Behaviour-Based Automatic Depression Analysis Using Hand-Crafted Statistics and Deep Learned Spectral Features

Cited by 93 publications

References 24 publications

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

The Human Touch: Using a Webcam to Autonomously Monitor Compliance During Visual Field Assessments

‘This time with feeling?’ Assessing EU data governance implications of out of home appraisal based emotional AI

Contact Info

Product

Resources

About