Analytical Validation of a Webcam-Based Assessment of Speech Kinematics: Digital Biomarker Evaluation following the V3 Framework

Abstract

<b><i>Introduction:</i></b> Kinematic analyses have recently revealed a strong potential to contribute to the assessment of neurological diseases. However, the validation of home-based kinematic assessments using consumer-grade video technology has yet to be performed. In line with best practices for digital biomarker development, we sought to validate webcam-based kinematic assessment against established, laboratory-based recording gold standards. We hypothesized that webcam-based kinematics would possess psychometric properties comparable to those obtained using the laboratory-based gold standards. <b><i>Methods:</i></b> We collected data from 21 healthy participants who repeated the phrase “buy Bobby a puppy” (BBP) at four different combinations of speaking rate and volume: Slow, Normal, Loud, and Fast. We recorded these samples twice back-to-back, simultaneously using (1) an electromagnetic articulography (“EMA”; NDI Wave) system, (2) a 3D camera (Intel RealSense), and (3) a 2D webcam for video recording via an in-house developed app. We focused on the extraction of kinematic features in this study, given their demonstrated value in detecting neurological impairments. We specifically extracted measures of speed/acceleration, range of motion (ROM), variability, and symmetry using the movements of the center of the lower lip during these tasks. Using these kinematic features, we derived measures of (1) agreement between recording methods, (2) test-retest reliability of each method, and (3) the validity of webcam recordings to capture expected changes in kinematics as a result of different speech conditions. <b><i>Results:</i></b> Kinematics measured using the webcam demonstrated good agreement with both the RealSense and EMA (ICC-A values often ≥0.70). Test-retest reliability, measured using the absolute agreement (2,1) formulation of the intraclass correlation coefficient (i.e., ICC-A), was often “moderate” to “strong” (i.e., ≥0.70) and similar between the webcam and EMA-based kinematic features. Finally, the webcam kinematics were typically as sensitive to differences in speech tasks as EMA and the 3D camera gold standards. <b><i>Discussion and Conclusions:</i></b> Our results suggested that webcam recordings display good psychometric properties, comparable to laboratory-based gold standards. This work paves the way for a large-scale clinical validation to continue the development of these promising technologies for the assessment of neurological diseases via home-based methods.

Improving Dysarthric Speech Segmentation With Emulated and Synthetic Augmentation

“…This sentence was selected due to its mix of phonetic components, including both voiced and unvoiced consonants, as well as high and low vowels and diphthongs. It is often used in dysarthria research as it offers some advantages in segmentation (i.e., has easily identifiable boundaries) [50], [51]. While our research focused on this single sentence to enable a clear and controlled comparison of the methodology's efficacy, we recognize the importance of generalizability in speech analysis.…”

“…This dataset was collected online and via a web application (App). More details regarding the complete App dataset can be found in [18].…”

“…A substantial barrier to the adoption of automated speech assessment, particularly in the case of remote assessments, is the considerable post-processing of data samples that is required prior to feature extraction. Given that speech assessments often involve repeating phrases and syllables [17], it is important to segment data into individual repetitions so that features can be extracted [18], [19], [20]. This "parsing" involves counting the repetitions, as well as identifying the onset and offset times of each repetition in the recording (audio or video data).…”

Development of webcam‐collected and artificial‐intelligence‐derived social and cognitive performance measures for neurodevelopmental genetic syndromes

“…Webcam data collection also permits the frame-by-frame automated facial expression analysis using machine learning algorithms that enable prototype matching using large training datasets. The potential for these methods to inform neurodevelopment is strong and, increasingly, both webcam-collected data (Simmatis et al, 2023) and artificial intelligence/machine learning algorithms (Nerusil et al, 2021) are being applied to create novel biometric measures for assessing child development and neurological conditions. A key advantage of webcam-based data collection is that the paradigms can be administered without direct real-time clinical supervision.…”