Explainable Depression Detection with Multi-Modalities Using a Hybrid Deep Learning Model on Social Media

Abstract

Social networks enable people to interact with one another by sharing information, sending messages, making friends, and having discussions, which generates massive amounts of data every day, popularly called as the user-generated content. This data is present in various forms such as images, text, videos, links, and others and reflects user behaviours including their mental states. It is challenging yet promising to automatically detect mental health problems from such data which is short, sparse and sometimes poorly phrased. However, there are efforts to automatically learn patterns using computational models on such user-generated content. While many previous works have largely studied the problem on a small-scale by assuming uni-modality of data which may not give us faithful results, we propose a novel scalable hybrid model that combines Bidirectional Gated Recurrent Units (BiGRUs) and Convolutional Neural Networks to detect depressed users on social media such as Twitter-based on multi-modal features. Specifically, we encode words in user posts using pre-trained word embeddings and BiGRUs to capture latent behavioural patterns, long-term dependencies, and correlation across the modalities, including semantic sequence features from the user timelines (posts). The CNN model then helps learn useful features. Our experiments show that our model outperforms several popular and strong baseline methods, demonstrating the effectiveness of combining deep learning with multi-modal features. We also show that our model helps improve predictive performance when detecting depression in users who are posting messages publicly on social media.

Mental Health Analysis in Social Media Posts: A Survey

“…Valence refers to the pleasant-unpleasant quality of a stimulus and ranges from negative to positive, whereas arousal refers to the intensity of a stimulus and ranges from dull to arousing. The past studies with MHA incorporate the Valence arousal dominance (VAD) Emotion model [36,39,43,74] and Plutchik model [7,75]. Plutchik's theory of emotion and emotional consequences for cognition, personality, and psychotherapy is derived from an evolutionary perspective [75].…”

“…6 Feature vector representation for mental health analysis in social media posts vectors. We witness existing works with attention mechanism such as Hierarchical Attention Mechanism (HAM) [32,105] to give importance to important posts for identifying suicidal tendencies [11,43]. The multi-attributed feature extraction is given as 3-level framework using three-level features extraction which consists of low level feature (linguistic features), middle level features (visual features) and high-level features (social features) to give as an input to the Deep Sparse Neural Network (DSNN) [76].…”

“…The observations about users' dynamic historical timeline on Twitter include improvements with interpretive Multi-Modal Depression Detection with Hierarchical Attention Network (MDHAN) [43]. The MDHAN framework is designed with multi-model features and two attention mechanisms are applied at tweet-level and at word-level, respectively.…”

“…6. Interpretability and Explainability: There are detailed and theoretical explanations of the proposed approach to test its interpretability [11] or explainability [2,7,43,83] via ablation studies. A complete section of ethi-cal validation must be explored further to enhance the applicability of the new methods in real-time applications.…”

“…These writings contain heterogeneous, unstructured and ill-formed data which is human-readable but difficult to interpret automatically by a system. Recent studies on predicting suicidal tendency on social media data by using machine learning, ML [1-5] models are more successful as compared to the medical records [6] and paved the way to explore deep learning, DL [7][8][9][10][11][12] and computational intelligence techniques [13] for quantifying suicidal tendency. We acknowledge that we limit the scope of our study to stress, clinical depression, and suicide risk.…”