Screening of Parkinsonian subtle fine-motor impairment from touchscreen typing via deep learning

Abstract

Fine-motor impairment (FMI) is progressively expressed in early Parkinson’s Disease (PD) patients and is now known to be evident in the immediate prodromal stage of the condition. The clinical techniques for detecting FMI may not be robust enough and here, we show that the subtle FMI of early PD patients can be effectively estimated from the analysis of natural smartphone touchscreen typing via deep learning networks, trained in stages of initialization and fine-tuning. In a validation dataset of 36,000 typing sessions from 39 subjects (17 healthy/22 PD patients with medically validated UPDRS Part III single-item scores), the proposed approach achieved values of area under the receiver operating characteristic curve (AUC) of 0.89 (95% confidence interval: 0.80–0.96) with sensitivity/specificity: 0.90/0.83. The derived estimations result in statistically significant ( ) correlation of 0.66/0.73/0.58 with the clinical standard UPDRS Part III items 22/23/31, respectively. Further validation analysis on 9 de novo PD patients vs. 17 healthy controls classification resulted in AUC of 0.97 (0.93–1.00) with 0.93/0.90. For 253 remote study participants, with self-reported health status providing 252.000 typing sessions via a touchscreen typing data acquisition mobile app (iPrognosis), the proposed approach predicted 0.79 AUC (0.66–0.91) with 0.76/0.71. Remote and unobtrusive screening of subtle FMI via natural smartphone usage, may assist in consolidating early and accurate diagnosis of PD.

Detection of Mild Cognitive Impairment Through Natural Language and Touchscreen Typing Processing

“…Before being tested to our dataset, consisting of more than 3,000 typing sessions of MCI patients and HC, the models have also been tested in 4 in-the-wild datasets: (1) a dataset with 216,000 typing sessions with keystroke dynamics from clinically examined 214 subjects (PD vs. HC), with self-reported demographics, through the iPROGNOSIS app, (2) a dataset with 36,000 typing sessions with keystroke dynamics from 39 subjects (PD/HC:22/17), (3) a subset of the previous dataset with 7,600 typing sessions, drawn from de novo PD patients and the same HC ( de novo PD/HC: 9/17), (4) the union of the first two datasets with 252,000 typing sessions with keystroke dynamics from 253 subjects (PD/HC: 67/186) The optimized scores produced indicators that can be used in-the-wild prediction of UPDRS scores 22/23/31, yielding correlation 0.66/0.73/0.58, respectively, in the validation set of 36,000 typing sessions. The trained models were used to infer based on the typing data from 11 MCI patients and 12 HC, and estimations for each symptom (R/B/AFT 0-4) were extracted for each typing session of each user ( 47 ). All the features from the natural language and typing processing can be found in Table 3 .…”

“…The in-the-clinic development dataset consists of 274 typing sessions in total (up to 10 text-excerpts for each user) with keystroke dynamics from 33 demographically matched subjects [18 early Parkinson Diseases (PD) patients and 15 HC], whom underwent clinical examination by neurologists and their FMI was evaluated by their UPDRS Part III single-item scores 22/23/31, expressing rigidity of upper extremity/alternate finger tapping/general body bradykinesia-hypokinesia, respectively. Before being tested to our dataset, consisting of more than 3,000 typing sessions of MCI patients and HC, the models have also been tested in 4 in-the-wild datasets: (1) a dataset with 216,000 typing sessions with keystroke dynamics from clinically examined 214 subjects (PD vs. HC), with self-reported demographics, through the iPROGNOSIS app, (2) a dataset with 36,000 typing sessions with keystroke dynamics from 39 subjects (PD/HC:22/17), (3) a subset of the previous dataset with 7,600 typing sessions, drawn from de novo PD patients and the same HC (de novo PD/HC: 9/17), (4) the union of the first two datasets with 252,000 typing sessions with keystroke dynamics from were extracted for each typing session of each user (47). All the features from the natural language and typing processing can be found in Table 3.…”

The state of telemedicine for persons with Parkinson's disease

“…Telemonitoring is the remote gathering of information about a patient which is used to inform healthcare providers (in a clinical setting) or researchers (in the framework of a trial). A wide and expanding spectrum of tools can be used for telemonitoring, including body-worn sensors [ 27 , 28 ], home sensors [ 29 ], specific apps for the smartphone [ 30 , 31 ], digital diaries [ 32 ], or analysis of common appliances such as computer keyboards [ 33 ] (only several selected high-quality references are given here). The promise of remote monitoring is to offer objective, continuous measures of relevant symptoms while patients are at home.…”

“…In a 6-year longitudinal study of prodromal individuals, specific gait characteristics such as step velocity and length were predictive of conversion to PD, even when measured as early as up to 4 years prior to the clinical diagnosis [ 40 ]. Other technologies suitable for early disease detection encompass touchscreen typing [ 31 ] or voice analysis [ 41 ]. However, voice studies often relied on high-quality data collected in controlled environments, making it difficult to apply such tools for large-scale screening based on less standardized real-life recordings.…”

Rapid development of an integrated remote programming platform for neuromodulation systems through the biodesign process

“…Current evidence on using remote monitoring with wearables, on-mobile sensors, and body-worn sensors to classify disease-related states is promising 10 , 31 , 32 . An integrated therapy application that is available on the patient’s mobile device is an opportunity to this data as it becomes integral to clinical decision-making in neuromodulation 33 . These data sets can be analyzed to further improve future therapeutics and to identify user-specific gaps and personalize care.…”