Estimating Severity of Depression From Acoustic Features and Embeddings of Natural Speech

Dumpala, Sri Harsha; Rempel, Sheri; Dikaios, Katerina; Sajjadian, Mehri; Uher, Rudolf; Oore, Sageev

doi:10.1109/icassp39728.2021.9414129

Cited by 12 publications

(9 citation statements)

References 24 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the TDNN x-vectors or F-TDNN x-vectors based on MFCC have demonstrated better performance than PLP i-vectors for the automatic detection of PD ( Moro-Velazquez et al, 2020 ). Besides, the x-vector technique has been used as an advanced method for emotion recognition ( Pappagari et al, 2020b ), AD detection ( Pappagari et al, 2020a ), and depression detection ( Dumpala et al, 2021 , 2022 ; Egas-López et al, 2022 ). Consequently, depression detection is carried out in this study using the x-vector approach with the i-vector framework as the baseline.…”

Section: Related Workmentioning

confidence: 99%

Ensemble learning with speaker embeddings in multiple speech task stimuli for depression detection

Liu

Li³

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

IntroductionAs a biomarker of depression, speech signal has attracted the interest of many researchers due to its characteristics of easy collection and non-invasive. However, subjects’ speech variation under different scenes and emotional stimuli, the insufficient amount of depression speech data for deep learning, and the variable length of speech frame-level features have an impact on the recognition performance.MethodsThe above problems, this study proposes a multi-task ensemble learning method based on speaker embeddings for depression classification. First, we extract the Mel Frequency Cepstral Coefficients (MFCC), the Perceptual Linear Predictive Coefficients (PLP), and the Filter Bank (FBANK) from the out-domain dataset (CN-Celeb) and train the Resnet x-vector extractor, Time delay neural network (TDNN) x-vector extractor, and i-vector extractor. Then, we extract the corresponding speaker embeddings of fixed length from the depression speech database of the Gansu Provincial Key Laboratory of Wearable Computing. Support Vector Machine (SVM) and Random Forest (RF) are used to obtain the classification results of speaker embeddings in nine speech tasks. To make full use of the information of speech tasks with different scenes and emotions, we aggregate the classification results of nine tasks into new features and then obtain the final classification results by using Multilayer Perceptron (MLP). In order to take advantage of the complementary effects of different features, Resnet x-vectors based on different acoustic features are fused in the ensemble learning method.ResultsExperimental results demonstrate that (1) MFCC-based Resnet x-vectors perform best among the nine speaker embeddings for depression detection; (2) interview speech is better than picture descriptions speech, and neutral stimulus is the best among the three emotional valences in the depression recognition task; (3) our multi-task ensemble learning method with MFCC-based Resnet x-vectors can effectively identify depressed patients; (4) in all cases, the combination of MFCC-based Resnet x-vectors and PLP-based Resnet x-vectors in our ensemble learning method achieves the best results, outperforming other literature studies using the depression speech database.DiscussionOur multi-task ensemble learning method with MFCC-based Resnet x-vectors can fuse the depression related information of different stimuli effectively, which provides a new approach for depression detection. The limitation of this method is that speaker embeddings extractors were pre-trained on the out-domain dataset. We will consider using the augmented in-domain dataset for pre-training to improve the depression recognition performance further.

show abstract

Section: Related Workmentioning

confidence: 99%

Ensemble learning with speaker embeddings in multiple speech task stimuli for depression detection

Liu

Li³

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The periodic training strategy defines the indefinite parameter ξ in the training rounds to be adjusted. The specific calculation of ξ is shown in Equation (16), where e n is the total number of training rounds, e i is the ith training round and f c is the adjustment factor.…”

Section: Periodic Focal Loss Functionmentioning

confidence: 99%

“…[15] combines a CNN network with a Growing Self‐organizing Map (GSOM) to identify psychological stress states based on spectral features. [16] proposes a multi‐task TCN learning model to estimate the degree of depression, combined with related tasks such as emotion and emotion recognition to identify depression. [17] proposes a self‐supervised acoustic feature extraction pre‐training network model for depression which uses a convolutional encoding‐decoding structure and combines it with the LSTM network for depression recognition.…”

Section: Introductionmentioning

confidence: 99%

Identification of depression state based on multi‐scale acoustic features in interrogation environment

Huang

Xiao

et al. 2023

IET Signal Processing

View full text Add to dashboard Cite

Depression diagnosis based on speech signals has the advantages of non-invasiveness, low cost, and few restrictions on portability. The research on the recognition of the depression state is carried out based on the acoustic information in the speech signal. Aiming at the interview dialogue speech in the consultation environment, a hierarchical attention temporal convolutional network (HATCN) acoustic depression recognition model is proposed. For sentence acoustic feature learning, a regional attention mechanism is introduced to extract multi-scale sentence features; for segment acoustic feature extraction, the traditional attention mechanism is used to calculate, which is in line with human cognitive mechanism. In addition, a periodic focal loss function is introduced to address the imbalance of positive and negative samples in depression diagnosis. Experiments show that the proposed acoustic depression recognition model has a certain improvement in recognition performance compared with other methods. At the same time, the influence of noise on the recognition of acoustic depression in the real consultation environment is analysed through experiments, and the data enhancement is carried out utilising speech noise, which proves the effectiveness of the data expansion of speech noise.

show abstract

“…DAIC-WOZ [12]. According to [49], even slight differences of the psychological state have the potential to cause a noticeable change in the acoustic domain, which makes audio data a competitive modality to be used as input to depression estimation frameworks [9,10,17,31,39,49,61]. Mel-cepstral features and formantfrequency tracks are two main arousal representations introduced by Williamson et al…”

Section: Related Workmentioning

confidence: 99%

Multi-modal Depression Estimation based on Sub-attentional Fusion

Wei¹,

Peng²,

Roitberg³

et al. 2022

Preprint

View full text Add to dashboard Cite

Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide. The information cues of depression can be harvested from diverse heterogeneous resources, e.g., audio, visual, and textual data, raising demand for new effective multi-modal fusion approaches for its automatic estimation. In this work, we tackle the task of automatically identifying depression from multi-modal data and introduce a sub-attention mechanism for linking heterogeneous information while leveraging Convolutional Bidirectional LSTM as our backbone. To validate this idea, we conduct extensive experiments on the public DAIC-WOZ benchmark for depression assessment featuring different evaluation modes and taking gender-specific biases into account. The proposed model yields effective results with 0.89 precision and 0.70 F1-score in detecting major depression and 4.92 MAE in estimating the severity. Our attention-based fusion module consistently outperforms conventional late fusion approaches and achieves a competitive performance compared to the previously published depression estimation frameworks, while learning to diagnose the disorder end-to-end and relying on far less preprocessing steps.

show abstract

Estimating Severity of Depression From Acoustic Features and Embeddings of Natural Speech

Cited by 12 publications

References 24 publications

Ensemble learning with speaker embeddings in multiple speech task stimuli for depression detection

Ensemble learning with speaker embeddings in multiple speech task stimuli for depression detection

Identification of depression state based on multi‐scale acoustic features in interrogation environment

Multi-modal Depression Estimation based on Sub-attentional Fusion

Contact Info

Product

Resources

About