Using Crowdsourced Listeners' Ratings to Measure Speech Changes in Hypokinetic Dysarthria: A Proof-of-Concept Study

Abstract

Purpose Interventions for speech disorders aim to produce changes that are not only acoustically measurable or perceptible to trained professionals but are also apparent to naive listeners. Due to challenges associated with obtaining ratings from suitably large listener samples, however, few studies currently evaluate speech interventions by this criterion. Online crowdsourcing technologies could enhance the measurement of intervention effects by making it easier to obtain real-world listeners' ratings. Method Stimuli, drawn from a published study by Sapir et al. (“Effects of intensive voice treatment (Lee Silverman Voice Treatment [LSVT]) on vowel articulation in dysarthric individuals with idiopathic Parkinson disease: Acoustic and perceptual findings” in Journal of Speech, Language, and Hearing Research, 50 (4), 2007), were words produced by individuals who received intensive treatment (LSVT LOUD) for hypokinetic dysarthria secondary to Parkinson's disease. Thirty-six online naive listeners heard randomly ordered pairs of words elicited pre- and posttreatment and reported which they perceived as “more clearly articulated.” Results Mixed-effects logistic regression indicated that words elicited posttreatment were significantly more likely to be rated “more clear.” Across individuals, acoustically measured magnitude of change was significantly correlated with pre–post difference in listener ratings. Conclusions These results partly replicate the findings of Sapir et al. (2007) and demonstrate that their acoustically measured changes are detectable by everyday listeners. This supports the viability of using crowdsourcing to obtain more functionally relevant measures of change in clinical speech samples. Supplemental Material https://doi.org/10.23641/asha.12170112

Acoustic vowel analysis and speech intelligibility in young adult Hebrew speakers: Developmental dysarthria versus typical development

“…Although recent evidence has demonstrated the efficacy of crowdsourced perceptual tasks completed on personal equipment (Nightingale et al . 2020), further studies may benefit from controlling such factors.…”

“…In order to increase participation of listeners, the intelligibility assessment was conducted on a Google‐based platform (see Nightingale et al . 2020 for a similar remote platform). Upon entry to this online task, listeners reported basic demographic information, and were presented with detailed instruction regarding the listening task.…”

“…Finally, intelligibility assessments were conducted via personal computer equipment, with limited control of factors concerning the listener transcriptions (e.g., stimulus intensity, background noise). Although recent evidence has demonstrated the efficacy of crowdsourced perceptual tasks completed on personal equipment (Nightingale et al 2020), further studies may benefit from controlling such factors.…”

Single Word Intelligibility of Individuals with Parkinson’s Disease in Noise: Pre-Specified Secondary Outcome Variables from a Randomized Control Trial (RCT) Comparing Two Intensive Speech Treatments (LSVT LOUD vs. LSVT ARTIC)

“…Several studies have investigated the impact of increased vocal intensity/loudness on speech intelligibility for people with PD in quiet listening conditions during un-cued speech tasks following intensive voice treatment [89][90][91]. In these studies, increased vocal loudness was found to increase intelligibility in PD participants for words extracted from read sentences [91], reading of passages [90], and conversational monologue [89].…”

Online-Crowdsourcing als „KommPaS“ in der kommunikationsbezogenen Dysarthriediagnostik

“…Für die Rekrutierung von Laienhörer(inne)n zur auditiven Bewertung von Sprechstörungen ist Online-Crowdsourcing eine vielversprechende Methode (Überblick in [5]). Das Verfahren wurde auch in der Dysarthrieforschung bereits erfolgreich eingesetzt [6]. Dabei konnte die Validität der Urteile im Vergleich zu Bewertungen unter Laborbedingungen nachgewiesen werden [7].…”