Emotion Recognition from Natural Phone Conversations in Individuals with and without Recent Suicidal Ideation

Gideon, John; Schatten, Heather T.; McInnis, Melvin G.; Provost, Emily Mower

doi:10.21437/interspeech.2019-1830

Cited by 21 publications

(11 citation statements)

References 24 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4.1, the best features in 2 out of 3 scenarios consist of behavior and turn-taking cues. Our findings are similar to those of previous works [11,12] which found affect-based and interaction dynamics cues to be important in identifying suicidal risk markers. This suggests that emotional regulation can serve as a useful source of features for these tasks.…”

Section: Classificationsupporting

confidence: 92%

Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations

Chakravarthula

Nasir

Tseng

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Suicide is a major societal challenge globally, with a wide range of risk factors, from individual health, psychological and behavioral elements to socio-economic aspects. Military personnel, in particular, are at especially high risk. Crisis resources, while helpful, are often constrained by access to clinical visits or therapist availability, especially when needed in a timely manner. There have hence been efforts on identifying whether communication patterns between couples at home can provide preliminary information about potential suicidal behaviors, prior to intervention. In this work, we investigate whether acoustic, lexical, behavior and turn-taking cues from military couples' conversations can provide meaningful markers of suicidal risk. We test their effectiveness in real-world noisy conditions by extracting these cues through an automatic diarization and speech recognition front-end. Evaluation is performed by classifying 3 degrees of suicidal risk: none, ideation, attempt. Our automatic system performs significantly better than chance in all classification scenarios and we find that behavior and turn-taking cues are the most informative ones. We also observe that conditioning on factors such as speaker gender and topic of discussion tends to improve classification performance.

show abstract

Section: Classificationsupporting

confidence: 92%

Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations

Chakravarthula

Nasir

Tseng

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…For instance, more than 50% of cases of post‐traumatic stress disorder (PTSD) co‐occur with depressive, anxiety, or substance use disorders . Furthermore, suicidal thoughts and behaviors can also be a goal in predictive models (for a review, see Reference ) and are present across many disorders. Comorbidity is one reason why the National Institute of Mental Health has developed the Research Domain Criteria with the goal of deconstructing diagnoses with biomarkers—from genetic to behavioral—to predict and improve response to treatments .…”

Section: Introductionmentioning

confidence: 99%

Automated assessment of psychiatric disorders using speech: A systematic review

Low

Bentley

Ghosh

2020

Laryngoscope Investig Oto

294

268

View full text Add to dashboard Cite

Objective There are many barriers to accessing mental health assessments including cost and stigma. Even when individuals receive professional care, assessments are intermittent and may be limited partly due to the episodic nature of psychiatric symptoms. Therefore, machine‐learning technology using speech samples obtained in the clinic or remotely could one day be a biomarker to improve diagnosis and treatment. To date, reviews have only focused on using acoustic features from speech to detect depression and schizophrenia. Here, we present the first systematic review of studies using speech for automated assessments across a broader range of psychiatric disorders. Methods We followed the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) guidelines. We included studies from the last 10 years using speech to identify the presence or severity of disorders within the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5). For each study, we describe sample size, clinical evaluation method, speech‐eliciting tasks, machine learning methodology, performance, and other relevant findings. Results 1395 studies were screened of which 127 studies met the inclusion criteria. The majority of studies were on depression, schizophrenia, and bipolar disorder, and the remaining on post‐traumatic stress disorder, anxiety disorders, and eating disorders. 63% of studies built machine learning predictive models, and the remaining 37% performed null‐hypothesis testing only. We provide an online database with our search results and synthesize how acoustic features appear in each disorder. Conclusion Speech processing technology could aid mental health assessments, but there are many obstacles to overcome, especially the need for comprehensive transdiagnostic and longitudinal studies. Given the diverse types of data sets, feature extraction, computational methodologies, and evaluation criteria, we provide guidelines for both acquiring data and building machine learning models with a focus on testing hypotheses, open science, reproducibility, and generalizability. Level of Evidence 3a

show abstract

“…The resulting predictions can then be used to bias response generation. Emotion classification is also used in mobile and web applications to identify heightened risk of suicidal ideation or mood fluctuations (Khorram et al ;Matton, McInnis, and Provost 2019;Gideon et al 2019), for the purpose of tracking or intervention. Data are sent from users' devices, including mobile applications (Khorram et al ) and Alexa or Google home devices (Piersol and Beddingfield 2019), and are stored on central servers for analysis.…”

Section: Introductionmentioning

confidence: 99%

Privacy Enhanced Multimodal Neural Representations for Emotion Recognition

Jaiswal

Provost

2020

AAAI

View full text Add to dashboard Cite

Many mobile applications and virtual conversational agents now aim to recognize and adapt to emotions. To enable this, data are transmitted from users' devices and stored on central servers. Yet, these data contain sensitive information that could be used by mobile applications without user's consent or, maliciously, by an eavesdropping adversary. In this work, we show how multimodal representations trained for a primary task, here emotion recognition, can unintentionally leak demographic information, which could override a selected opt-out option by the user. We analyze how this leakage differs in representations obtained from textual, acoustic, and multimodal data. We use an adversarial learning paradigm to unlearn the private information present in a representation and investigate the effect of varying the strength of the adversarial component on the primary task and on the privacy metric, defined here as the inability of an attacker to predict specific demographic information. We evaluate this paradigm on multiple datasets and show that we can improve the privacy metric while not significantly impacting the performance on the primary task. To the best of our knowledge, this is the first work to analyze how the privacy metric differs across modalities and how multiple privacy concerns can be tackled while still maintaining performance on emotion recognition.

show abstract

Emotion Recognition from Natural Phone Conversations in Individuals with and without Recent Suicidal Ideation

Cited by 21 publications

References 24 publications

Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations

Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations

Automated assessment of psychiatric disorders using speech: A systematic review

Privacy Enhanced Multimodal Neural Representations for Emotion Recognition

Contact Info

Product

Resources

About