Machine Learning-based Voice Assessment for the Detection of Positive and Recovered COVID-19 Patients

Robotti, Carlo; Costantini, Giovanni; Saggio, Giovanni; Cesarini, Valerio; Calastri, Anna; Maiorano, Eugenia; Piloni, Davide; Perrone, Tiziano; Sabatini, Umberto; Cassaniti, Irene; Baldanti, Fausto; Gravina, Andrea; Sakib, Ahmed Shahriar; Alessi, Elena; Pascucci, Matteo; Casali, Daniele; Zarezadeh, Zakarya; Zoppo, Vincenzo Del; Pisani, Antonio; Benazzo, Marco

doi:10.1016/j.jvoice.2021.11.004

Cited by 21 publications

(11 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the last three have somehow been often used in the literature, the usage of RASTA is still seminal and underrated. However, RASTA is a frequency-of-frequency kind of filtering, based on an all-pole model, which is inherently noise-robust and insensitive to slowly varying spectral components, and often improves classification performances for speech tasks with respect to similar domains [ 69 , 70 , 71 , 72 ].…”

Section: Discussionmentioning

confidence: 99%

The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning

Costantini

Parada-Cabaleiro

Casali

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Machine Learning (ML) algorithms within a human–computer framework are the leading force in speech emotion recognition (SER). However, few studies explore cross-corpora aspects of SER; this work aims to explore the feasibility and characteristics of a cross-linguistic, cross-gender SER. Three ML classifiers (SVM, Naïve Bayes and MLP) are applied to acoustic features, obtained through a procedure based on Kononenko’s discretization and correlation-based feature selection. The system encompasses five emotions (disgust, fear, happiness, anger and sadness), using the Emofilm database, comprised of short clips of English movies and the respective Italian and Spanish dubbed versions, for a total of 1115 annotated utterances. The results see MLP as the most effective classifier, with accuracies higher than 90% for single-language approaches, while the cross-language classifier still yields accuracies higher than 80%. The results show cross-gender tasks to be more difficult than those involving two languages, suggesting greater differences between emotions expressed by male versus female subjects than between different languages. Four feature domains, namely, RASTA, F0, MFCC and spectral energy, are algorithmically assessed as the most effective, refining existing literature and approaches based on standard sets. To our knowledge, this is one of the first studies encompassing cross-gender and cross-linguistic assessments on SER.

show abstract

Section: Discussionmentioning

confidence: 99%

The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning

Costantini

Parada-Cabaleiro

Casali

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similar COVID-19 infection risk evaluation systems and large cough databases are currently under development [27], but additional research still needs to be performed to assess whether such algorithms predict the true COVID-19 status or rather the general health status of the individual [28]. We can also mention the recent PLOS DIGITAL HEALTH [29]. In one sensitivity analysis, they have trained algorithms to classify positive versus recovered individuals, which is the closest approach to ours.…”

Section: Comparison With the Literaturementioning

confidence: 99%

A voice-based biomarker for monitoring symptom resolution in adults with COVID-19: Findings from the prospective Predi-COVID cohort study

Fagherazzi

Zhang²,

Elbéji³

et al. 2022

PLOS Digit Health

View full text Add to dashboard Cite

People with COVID-19 can experience impairing symptoms that require enhanced surveillance. Our objective was to train an artificial intelligence-based model to predict the presence of COVID-19 symptoms and derive a digital vocal biomarker for easily and quantitatively monitoring symptom resolution. We used data from 272 participants in the prospective Predi-COVID cohort study recruited between May 2020 and May 2021. A total of 6473 voice features were derived from recordings of participants reading a standardized pre-specified text. Models were trained separately for Android devices and iOS devices. A binary outcome (symptomatic versus asymptomatic) was considered, based on a list of 14 frequent COVID-19 related symptoms. A total of 1775 audio recordings were analyzed (6.5 recordings per participant on average), including 1049 corresponding to symptomatic cases and 726 to asymptomatic ones. The best performances were obtained from Support Vector Machine models for both audio formats. We observed an elevated predictive capacity for both Android (AUC = 0.92, balanced accuracy = 0.83) and iOS (AUC = 0.85, balanced accuracy = 0.77) as well as low Brier scores (0.11 and 0.16 respectively for Android and iOS when assessing calibration. The vocal biomarker derived from the predictive models accurately discriminated asymptomatic from symptomatic individuals with COVID-19 (t-test P-values<0.001). In this prospective cohort study, we have demonstrated that using a simple, reproducible task of reading a standardized pre-specified text of 25 seconds enabled us to derive a vocal biomarker for monitoring the resolution of COVID-19 related symptoms with high accuracy and calibration.

show abstract

“…Pinkas et al (2020) utilized sustained phonemes and a counting task to achieve an AUC of 0.81 when models formed from all of the tasks were combined into an ensemble model 13 . Saggio et al (2021) utilized features from sustained vowels to discriminate between positive COVID-19 patients and healthy controls with an AUC of 0.94, and between positive -COVID-19 patients and recovered negative COVID-19 individuals with an AUC of 0.97 14 . While there has been promise in using these acoustic recordings, many of them utilize deep learning techniques that do not easily reveal the most important features, which could be used to better understand the physiological underpinnings of how speech is affected during COVID-19 and hopefully help generalize to future samples.…”

Section: Introductionmentioning

confidence: 99%

Dissociating COVID-19 from other respiratory infections based on acoustic, motor coordination, and phonemic patterns

Talkar

Low

Simpkin

et al. 2023

Sci Rep

View full text Add to dashboard Cite

In the face of the global pandemic caused by the disease COVID-19, researchers have increasingly turned to simple measures to detect and monitor the presence of the disease in individuals at home. We sought to determine if measures of neuromotor coordination, derived from acoustic time series, as well as phoneme-based and standard acoustic features extracted from recordings of simple speech tasks could aid in detecting the presence of COVID-19. We further hypothesized that these features would aid in characterizing the effect of COVID-19 on speech production systems. A protocol, consisting of a variety of speech tasks, was administered to 12 individuals with COVID-19 and 15 individuals with other viral infections at University Hospital Galway. From these recordings, we extracted a set of acoustic time series representative of speech production subsystems, as well as their univariate statistics. The time series were further utilized to derive correlation-based features, a proxy for speech production motor coordination. We additionally extracted phoneme-based features. These features were used to create machine learning models to distinguish between the COVID-19 positive and other viral infection groups, with respiratory- and laryngeal-based features resulting in the highest performance. Coordination-based features derived from harmonic-to-noise ratio time series from read speech discriminated between the two groups with an area under the ROC curve (AUC) of 0.94. A longitudinal case study of two subjects, one from each group, revealed differences in laryngeal based acoustic features, consistent with observed physiological differences between the two groups. The results from this analysis highlight the promise of using nonintrusive sensing through simple speech recordings for early warning and tracking of COVID-19.

show abstract

Machine Learning-based Voice Assessment for the Detection of Positive and Recovered COVID-19 Patients

Cited by 21 publications

References 95 publications

The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning

The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning

A voice-based biomarker for monitoring symptom resolution in adults with COVID-19: Findings from the prospective Predi-COVID cohort study

Dissociating COVID-19 from other respiratory infections based on acoustic, motor coordination, and phonemic patterns

Contact Info

Product

Resources

About