AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition

Ringeval, Fabien; Schuller, Björn; Valstar, Michel; Cummins, Nicholas; Cowie, Roddy; Tavabi, Leili; Schmitt, Maximilian; Alisamir, Sina; Amiriparian, Shahin; Meßner, Eva-Maria; Song, Siyang; Liu, Shuo; Zhao, Ziping; Mallol-Ragolta, Adria; Ren, Zhao; Soleymani, Mohammad; Pantić, Maja

doi:10.1145/3347320.3357688

Cited by 249 publications

(235 citation statements)

References 60 publications

Supporting

Mentioning

231

Contrasting

Order By: Relevance

“…Interestingly, interviews may be done by virtual humans or avatars which reduces costs, may increase comfortableness for some participants, and this can help re-enact dramatic scenarios the same way across users, [119][120][121] which has been used in the AVEC challenges for depression classification. 8…”

Section: Use Multiple Tasksmentioning

confidence: 99%

Automated assessment of psychiatric disorders using speech: A systematic review

Low

Bentley

Ghosh

2020

Laryngoscope Investig Oto

250

268

View full text Add to dashboard Cite

Objective There are many barriers to accessing mental health assessments including cost and stigma. Even when individuals receive professional care, assessments are intermittent and may be limited partly due to the episodic nature of psychiatric symptoms. Therefore, machine‐learning technology using speech samples obtained in the clinic or remotely could one day be a biomarker to improve diagnosis and treatment. To date, reviews have only focused on using acoustic features from speech to detect depression and schizophrenia. Here, we present the first systematic review of studies using speech for automated assessments across a broader range of psychiatric disorders. Methods We followed the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) guidelines. We included studies from the last 10 years using speech to identify the presence or severity of disorders within the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5). For each study, we describe sample size, clinical evaluation method, speech‐eliciting tasks, machine learning methodology, performance, and other relevant findings. Results 1395 studies were screened of which 127 studies met the inclusion criteria. The majority of studies were on depression, schizophrenia, and bipolar disorder, and the remaining on post‐traumatic stress disorder, anxiety disorders, and eating disorders. 63% of studies built machine learning predictive models, and the remaining 37% performed null‐hypothesis testing only. We provide an online database with our search results and synthesize how acoustic features appear in each disorder. Conclusion Speech processing technology could aid mental health assessments, but there are many obstacles to overcome, especially the need for comprehensive transdiagnostic and longitudinal studies. Given the diverse types of data sets, feature extraction, computational methodologies, and evaluation criteria, we provide guidelines for both acquiring data and building machine learning models with a focus on testing hypotheses, open science, reproducibility, and generalizability. Level of Evidence 3a

show abstract

Section: Use Multiple Tasksmentioning

confidence: 99%

Automated assessment of psychiatric disorders using speech: A systematic review

Low

Bentley

Ghosh

2020

Laryngoscope Investig Oto

250

268

View full text Add to dashboard Cite

show abstract

“…The choice of the loss function is frequently determined by the metric used for evaluation. In the case of dimensional emotion recognition, Ringeval et al of proposed the use of a concordance correlation coefficient (CCC) to score the performance of predicted emotion attributes [3].…”

Section: Related Workmentioning

confidence: 99%

“…Ringeval et al provides baseline fusion method for late fusion strategy from SEWA dataset [3,12]. The results from each modality or feature set can be combined using a static regressor, i.e., SVR to make the final decision of predicted emotion attribute scores from given results of several modalities.…”

Section: Related Workmentioning

confidence: 99%

“…The SEWA dataset [12] provided in [3] is used in this research. The dataset contains audiovisual recordings from: Chinese, English, German, Greek, Hungarian, and Serbian, but only German (DE) and Hungarian (HU) are used in this work as they did not provide test label in other languages.…”

Section: Data and Feature Setsmentioning

confidence: 99%

“…In this paper, we evaluated three emotional dimensions/attributes: arousal, valence, and liking, which have been obtained from the dataset in [3]. The task is to obtain the most accurate prediction on a specific metric.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multitask Learning and Multistage Fusion for Dimensional Audiovisual Emotion Recognition

Atmaja

Akagi

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Due to its ability to accurately predict emotional state using multimodal features, audiovisual emotion recognition has recently gained more interest from researchers. This paper proposes two methods to predict emotional attributes from audio and visual data using a multitask learning and a fusion strategy. First, multitask learning is employed by adjusting three parameters for each attribute to improve the recognition rate. Second, a multistage fusion is proposed to combine results from various modalities' final prediction. Our approach used multitask learning, employed at unimodal and early fusion methods, shows improvement over single-task learning with an average CCC score of 0.431 compared to 0.297. A multistage method, employed at the late fusion approach, significantly improved the agreement score between true and predicted values on the development set of data (from [0.537, 0.565, 0.083] to [0.68, 0.656, 0.443]) for arousal, valence, and liking.

show abstract