A machine learning perspective on the emotional content of Parkinsonian speech

Sechidis, Konstantinos; Fusaroli, Riccardo; Orozco-Arroyave, Juan Rafael; Wolf, Detlef; Zhang, Yanping

doi:10.1016/j.artmed.2021.102061

Cited by 20 publications

(29 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…different languages or recording conditions, and combine their knowledge at inference time hereby improving generalizability 26 . In a previous study, using different SER corpora for external validation, the MoE model outperformed all constituent models as well as a single model trained on the pooled data from all the different corpora 20 . Further details on feature extraction, training, and validation of the SER model are provided in the Supplementary Material, hence the following sections in Materials and Methods relate to the depression corpus.…”

Section: Methodsmentioning

confidence: 85%

“…The SER model was trained following Sechidis et al 20 on the public CREMA-D 21 , RAVDESS 22 , and EMO-DB 23 datasets, all of which consist of recordings of sentences spoken by professional actors portraying different emotions. CREMA-D and RAVDESS include American English speech and EMO-DB German speech.…”

Section: Methodsmentioning

confidence: 99%

“…We made use of the model trained in Sechidis et al 1 for SER and briefly describe it here for completeness. The SER model was trained on the CREMA-D 2 , RAVDESS 3 , and EMO-DB 4 datasets and validated on EMOVO 5 , TESS 6 , and SAVEE 7 .…”

Section: Speech Emotion Recognition With Mixture Of Expertsmentioning

confidence: 99%

See 2 more Smart Citations

A Generalizable Speech Emotion Recognition Model Reveals Depression and Remission

Hansen

Zhang

Wolf

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Objective: Affective disorders have long been associated with atypical voice patterns, however, current work on automated voice analysis often suffers from small sample sizes and untested generalizability. This study investigated a generalizable approach to aid clinical evaluation of depression and remission from voice. Methods: A Mixture-of-Experts machine learning model was trained to infer happy/sad emotional state using three publicly available emotional speech corpora. We examined the model's predictive ability to classify the presence of depression on Danish speaking healthy controls (N = 42), patients with first-episode major depressive disorder (MDD) (N = 40), and the same patients in remission (N = 25) based on recorded clinical interviews. The model was evaluated on raw data, data cleaned for background noise, and speaker diarized data. Results: The model showed reliable separation between healthy controls and depressed patients at the first visit, obtaining an AUC of 0.71. Further, we observed a reliable treatment effect in the depression group, with speech from patients in remission being indistinguishable from that of the control group. Model predictions were stable throughout the interview, suggesting that as little as 20-30 seconds of speech is enough to accurately screen a patient. Background noise (but not speaker diarization) heavily impacted predictions, suggesting that a controlled environment and consistent preprocessing pipelines are crucial for correct characterizations. Conclusion: A generalizable speech emotion recognition model can effectively reveal changes in speaker depressive states before and after treatment in patients with MDD. Data collection settings and data cleaning are crucial when considering automated voice analysis for clinical purposes.

show abstract

Section: Methodsmentioning

confidence: 85%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Generalizable Speech Emotion Recognition Model Reveals Depression and Remission

Hansen

Zhang

Wolf

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…When the user finishes recording, the system will save the speech information input by the user as a speech file. Figure 4 shows the flow chart of speech acquisition [11]. 6 shows the specific flow chart of MFCC feature extraction [13].…”

Section: Design and Implementation Of Speech Recognitionmentioning

confidence: 99%

A Spoken English Teaching System Based on Speech Recognition and Machine Learning

Jiao

Song

Zhao

et al. 2021

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

The learning model and environment are two major constraints on spoken English learning by Chinese learners. The maturity of computer-aided language learning brings a new opportunity to spoken English learners. Based on speech recognition and machine learning, this paper designs a spoken English teaching system, and determines the overall architecture and functional modules of the system according to the system’s functional demand. Specifically, MATLAB was adopted to realize speech recognition, and generate a speech recognition module. Combined with machine learning algorithm, a deep belief network (DBN)-support vector machine (SVM) model was proposed to classify and detect the errors in pronunciation; the module also scores the quality and corrects the errors in pronunciation. This model was extended to a speech evaluation module was created. Next, several experiments were carried out to test multiple attributes of the system, including the accuracy of pronunciation classification and error detection, recognition rates of different environments and vocabularies, and the real-timeliness of recognition. The results show that our system achieved good performance, realized the preset design goals, and satisfied the user demand. This research provides an important theoretical and practical reference to transforming English teaching method, and improving the spoken English of learners.

show abstract

“…While it is well accepted that autistic individuals often have atypical voices -e.g., sing-songy or monotone intonation and what has been referred to as inappropriate prosody (Baltaxe & Simmons, 1985;McCann & Peppé, 2003;Patel et al, 2019) -, acoustic investigations of the physical properties of voice underlying such perceptions often present weak or inconsistent findings. A recent meta-analysis identified increased and more variable fundamental frequency, as well as more and longer pauses as common characteristics (Fusaroli et al, 2017), these differences were, however, small and could only be partially replicated on new samples, across biological sexes, and languages (Fusaroli et al, 2018(Fusaroli et al, , 2021. Several ML studies have tried complementary approaches to these piecewise, highly top-down approaches.…”

Section: Introductionmentioning

confidence: 99%

Vocal markers of autism: assessing the generalizability of machine learning models

Rybner

Jessen

Mortensen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Machine learning (ML) approaches show increasing promise in their ability to identify vocal markers of autism. Nonetheless, it is unclear to what extent such markers generalize to new speech samples collected e.g., using a different speech task or in a different language. In this paper, we systematically assess the generalizability of ML findings across a variety of contexts. We train promising published ML models of vocal markers of autism on novel cross-linguistic datasets following a rigorous pipeline to minimize overfitting, including cross-validated training and ensemble models. We test the generalizability of the models by testing them on i) different participants from the same study, performing the same task; ii) the same participants, performing a different (but similar) task; iii) a different study with participants speaking a different language, performing the same type of task.While model performance is similar to previously published findings when trained and tested on data from the same study (out-of-sample performance), there is considerable variance between studies. Crucially, the models do not generalize well to different, though similar, tasks and not at all to new languages. The ML pipeline is openly shared.Generalizability of ML models of vocal markers - and more generally biobehavioral markers - of autism is an issue. We outline three recommendations for strategies researchers could take to be more explicit about generalizability and improve it in future studies.

show abstract

A machine learning perspective on the emotional content of Parkinsonian speech

Cited by 20 publications

References 35 publications

A Generalizable Speech Emotion Recognition Model Reveals Depression and Remission

A Generalizable Speech Emotion Recognition Model Reveals Depression and Remission

A Spoken English Teaching System Based on Speech Recognition and Machine Learning

Vocal markers of autism: assessing the generalizability of machine learning models

Contact Info

Product

Resources

About