Things to Consider When Automatically Detecting Parkinson’s Disease Using the Phonation of Sustained Vowels: Analysis of Methodological Issues

Abstract

Diagnosing Parkinson’s Disease (PD) necessitates monitoring symptom progression. Unfortunately, diagnostic confirmation often occurs years after disease onset. A more sensitive and objective approach is paramount to the expedient diagnosis and treatment of persons with PD (PwPDs). Recent studies have shown that we can train accurate models to detect signs of PD from audio recordings of confirmed PwPDs. However, disparities exist between studies and may be caused, in part, by differences in employed corpora or methodologies. Our hypothesis is that unaccounted covariates in methodology, experimental design, and data preparation resulted in overly optimistic results in studies of PD automatic detection employing sustained vowels. These issues include record-wise fold creation rather than subject-wise; an imbalance of age between the PwPD and control classes; using too small of a corpus compared to the sizes of feature vectors; performing cross-validation without including development data; and the absence of cross-corpora testing to confirm results. In this paper, we evaluate the influence of these methodological issues in the automatic detection of PD employing sustained vowels. We perform several experiments isolating each issue to measure its influence employing three different corpora. Moreover, we analyze if the perceived dysphonia of the speakers could be causing differences in results between the corpora. Results suggest that each independent methodological issue analyzed has an effect on classification accuracy. Consequently, we recommend a list of methodological steps to be considered in future experiments to avoid overoptimistic or misleading results.

Exploring Digital Speech Biomarkers of Hypokinetic Dysarthria in a Multilingual Cohort

“…The drop in accuracy when classifying people speaking a different language to subjects used during training is consistent with previous studies’ results [22, 23, 25], but it appears that data from another carefully-selected language can be used to improve the model performance. From the importance coefficients of each model (Figure 4), we can observe that in the classification, features are differently important for each scenario, which confirms the earlier findings [28]. Counting the occurrences of features in all feature importance scenarios, the best results are provided by the feature [a]-relNAQSD, which is important in the model trained on all languages and in four out of five separate models.…”

“…From the importance coefficients of each model (Figure 4), we can observe that in the classification, features are differently important for each scenario, which confirms the earlier findings [28]. Counting the occurrences of features in all feature importance scenarios, the best results are provided by the feature [a]-relNAQSD, which is important in the model trained on all languages and in four out of five separate models.…”

Automated gender‐Parkinson's disease detection at the same time via a hybrid deep model using human voice

“…Also, as used in the literature, cross validation prevents memorization. [42][43][44] So, for examining success rate feature set is divided into training and test set with 12-fold cross validation. Since this data composed of four class label, it means of multi-class classification.…”

“…mRMR feature selection method is chosen to increase success rate. Also, as used in the literature, cross validation prevents memorization 42‐44 . So, for examining success rate feature set is divided into training and test set with 12‐fold cross validation.…”

Hybrid Machine Learning Framework for Multistage Parkinson’s Disease Classification Using Acoustic Features of Sustained Korean Vowels

“…As a result, the analysis of speech signals offers a greater possibility of detecting Parkinson’s in its early stages, and speech analysis can be used as a non-invasive and cost-effective tool in the early detection and monitoring of PD [ 10 ]. Recent studies on PD telediagnosis have focused on identifying vocal impairments through sustained vowel phonation, or running speech, in subjects [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. These studies have employed various speech-signal-processing algorithms to extract clinically relevant data for the assessment of PD.…”