Spectral Representation of Behaviour Primitives for Depression Analysis

Abstract

Depression is a serious mental disorder affecting millions of people all over the world. Traditional clinical diagnosis methods are subjective, complicated and require extensive participation of clinicians. Recent advances in automatic depression analysis systems promise a future where these shortcomings are addressed by objective, repeatable, and readily available diagnostic tools to aid health professionals in their work. Yet there remain a number of barriers to the development of such tools. One barrier is that existing automatic depression analysis algorithms base their predictions on very brief sequential segments, sometimes as little as one frame. Another barrier is that existing methods do not take into account what the context of the measured behaviour is. In this paper, we extract multi-scale video-level features for video-based automatic depression analysis. We propose to use automatically detected human behaviour primitives as the low-dimensional descriptor for each frame. We also propose two novel spectral representations, i.e. spectral heatmaps and spectral vectors, to represent video-level multi-scale temporal dynamics of expressive behaviour. Constructed spectral representations are fed to Convolution Neural Networks (CNNs) and Artificial Neural Networks (ANNs) for depression analysis. We conducted experiments on the AVEC 2013 and AVEC 2014 benchmark datasets to investigate the influence of interview tasks on depression analysis. In addition to achieving state of the art accuracy in severity of depression estimation, we show that the task conducted by the user matters, that fusion of a combination of tasks reaches highest accuracy, and that longer tasks are more informative than shorter tasks, up to a point.

Design and Evaluation of Virtual Human Mediated Tasks for Assessment of Depression and Anxiety

“…Existing works have focused on either optimising the social abilities (e.g., speech recognition and dialogue management) and emotional expressiveness (e.g., empathy) of the virtual human to induce trust and natural behaviour of users [5,11,12] or evaluating its reliability as an assessment tool [15,21], whereas less attention has been given to investigating the impact of virtual human mediation on the user's behaviour expressiveness. Furthermore, although the activity performed by a user influences the predictive ability of the automatic depression and anxiety estimation systems [31]…”

“…Regarding the relatively higher head yaw (left-right head turn) observed in the VH group, it is also possible that system design artefacts could have influenced head activity, e.g., looking from the on-screen text to the virtual human, but this needs to be confirmed empirically with further analysis of the eye gaze data. The performance of detection systems for depression and anxiety levels has advanced lately due to their improved ability to detect distinctive verbal & non-verbal behaviour associated with these conditions [31]. VH-mediated systems capable of evoking these characteristics could help fast-track the development of personalised digital interventions that would promote treatment accessibility to a broader population.…”

“…Recent advances in social signal processing and affective computing have enabled automatic measurements of many aspects of these biomarkers of human behaviour, and these have been utilised for objective prediction of depression and anxiety levels, with little human intervention [32]. Further, [31] demonstrated that the performance of such predictive systems on mental health assessment depends on what task a user is performing whilst being observed. However, there is still an open question of what an optimal task is and what characterises an optimal task.…”

Personality Recognition by Modelling Person-specific Cognitive Processes using Graph Representation

“…The problem with these approaches is that at the level of a single frame or short segment, even people with different personality traits may display very similar non-verbal audio-visual behaviours. Therefore, these training strategies would end up utilising the same input pattern with multiple labels, making it practically impossible to train a model that has a good generalization capability [44,45,47] (Problem 2). Although some approaches select a set of key frames to represent an entire video and infer personality from such video-level representations [4,29,53] , they ignore the details contained in the discarded frames (Problem 3).…”

Separable convolutional neural networks for facial expressions recognition

“…Facial Expressions Recognition is paramount to build an affective system. The system can be implemented in several application such as, but not limited to: medical area (e.g., depression analysis [2], nervous system disorder [3]), entertainment area (e.g., games [4,5]), virtual humans/agents or conversational agents [4,6,7] and many more. Several research efforts have focused on building an automatic facial expressions recognition system, and there remain some challenges yet to be solved.…”