PHQ-V/GAD-V: Assessments to Identify Signals of Depression and Anxiety from Patient Video Responses

Abstract

Mental health issues are a growing problem worldwide, and their detection can be complicated. Assessments such as the Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder (GAD-7) questionnaire can be useful tools for detecting depression and anxiety, however, due to being self-reported, patients may underestimate their own risk. To address this problem, two new assessments are introduced, i.e., the PHQ-V and GAD-V, that utilize open-ended video questions adapted from the PHQ-9 and GAD-7 assessments. These video-based assessments analyze language, audio, and facial features by applying recent work in machine learning, namely pre-trained transformer networks, to provide an additional source of information for detecting risk of illness. The PHQ-V and GAD-V are adept at predicting the original PHQ-9 and GAD-7 scores. Analysis of their errors shows that they can detect depression and anxiety in even cases where the self-reported assessments fail to do so. These assessments provide a valuable new set of tools to help detect risk of depression and anxiety.

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

“…Modality and Feature Comparisons Most studies on multimodal detection justified the effectiveness of combining multiple modalities due to their complementary outcomes, which outperformed unimodal approaches. Notably, we noticed a single exception in a finding [174] that textual modality alone is succinctly effective, such that combining it with audio and visual modalities slightly deteriorated the overall performance. Deeper analyses also revealed that specific modalities could be more influential than others.…”

“…On the other hand, several other studies adopted neural networks to enforce the exact dimensions of multimodal representations. For example, using fully connected (FC) layers with the same units to condense features to a uniform dimension [ 70 , 112 , 179 ], a multilayer perceptron (MLP) [ 111 ], or bidirectional gated recurrent unit (Bi-GRU) [ 106 ] to transform representations to specific dimensions and a custom transformer-based architecture that applies linear projection to match various representation sizes [ 174 ]. In addition, an FC layer was also used to embed categorical variables to be concatenated with continuous variables [ 170 ].…”

“…com/affdex-for-market-research (accessed on 10 December 2023), DBFace https: //github.com/dlunion/DBFace (accessed on 10 December 2023), FaceMesh https://developers. google.com/android/reference/ com/google/mlkit/vision/ facemesh/FaceMesh (accessed on 10 December 2023), dlib [347] [ 25,53,55,56,68,79,80,83,91,92,94,98,99,101,108,174,193,199,201,203,220 OpenFace [269], Face++ [352],…”

“…Learn shared representations from weighted modality-specific representations Gated Multimodal Unit (GMU) [429], parallel attention model, attention layer, sparse MLP (mix vertical and horizontal information via weight sharing and sparse connection), multimodal encoder-decoder, multimodal factorized bilinear pooling (combines compact output features of multi-modal low-rank bilinear [430] and robustness of multi-modal compact bilinear [431]), multi-head intermodal attention fusion, transformer [295], feed-forward network, low-rank multimodal fusion network [432] [ 62,65,67,76,93,100,102,106,113,117,131,135,136,[142][143][144]174,218,433] Learn joint sparse representations Dictionary learning [20] Learn and fuse outputs from different modality-specific parts at fixed time steps Cell-coupled LSTM with L-skip fusion mechanism [101] Learn cross-modality representations that incorporate interactions between modalities LXMERT [434], transformer encoder with cross-attention layers (representations of a modality as query and the other as key/value, and vice versa), memory fusion network [435] [82, 92,129] Horizontal and vertical kernels to capture patterns across different levels CASER [309] [170]…”

“…Fully connected (FC) layers, multilayer perceptron (MLP), CNN, LSTM, BiLSTM, GRU, temporal convolutional network (TCN) [441] (with dilation for long sequences)-with activation function like Sigmoid, Softmax, ReLU, LeakyReLU, and GeLU Predict scores of assessment scales (regression) or probability distribution over classes (classification) [60,78,80,[84][85][86][87][88][90][91][92][93][94]96,98,105,111,113,117,131,133,135,136,[142][143][144]146,162,163,167,168,170,172,174,178,179,190,197,199,201,218,219,221,223,308] DCNN-DNN (combination of deep CNN and DNN), GCNN-LSTM (combination of gated convolutional neural network, which replaces a convolution block in CNN wi...…”