Detecting MCI using real‐time, ecologically valid data capture methodology: How to improve scientific rigor in digital biomarker analyses

Abstract

Background Early identification and accurate assessment of Mild Cognitive Impairment (MCI) is critical for clinical‐trial enrichment as well as the early intervention of the neurodegenerative disease. Continuous home‐based measurements of functions using simple embedded sensors and devices could provide an opportunity to improve the sensitivity and specificity in identifying MCI subjects in the community. However, a large number of assessment data points from each individual might increase the possibility of a chance finding. Careful and creative approaches are required to confirm robustness of the findings. Methods A total of 152 participants enrolled in the Intelligent Systems for Assessing Aging Changes Study (ISAAC) from the Oregon Center for Aging and Technology (ORCATECH) were used in the current project. In‐home monitored functional measurements including daily average walking speed, walking variability, computer usage and other functional measures were used. Each subject contributed 1372 days of data on average (SD = 662.3). In order to reflect the fact that various stages of MCI and NC are enrolled into trials in real‐world, longitudinal trajectories from each participant was sliced into small pieces, i.e., slicing windows of different sizes and used for cross‐validation. The cross‐validation was conducted on randomly permutated left‐out validation data to evaluate the performance. We also examined proportion of missingness during each slicing window and examined its impact on the ability in identifying MCI. We used various feature selection methods including F1‐score, mutual information, and chi2 to rank the effectiveness of variables. From the selected variables, we constructed different machine learning models, including logistic regression and support vector machines. Results Models using top‐4 features gave the best ability in differentiating MCI from NC, indicated by Receiver Operating Characteristics Area under Curve. Using slicing windows with a missing ratio higher than 30% was found to compromise the model performance. Longer subsequences have more information for modeling, but subsequences longer than a 2‐month window did not further help the performance. Conclusions This study proposed an approach that effectively detects MCI from in‐home monitoring measurement, which could benefit study enrichment in preclinical trails. The approaches presented could aid in improving scientific rigor of digital biomarker analyses.

Redefining and Validating Digital Biomarkers as Fluid, Dynamic Multi-Dimensional Digital Signal Patterns

“…The second month may include many steps throughout the house (e.g., indicator of searching for a misplaced item), digital voice patterns of narrower range of word choice in the home environment, and an instance of fast paced steps to the kitchen near the area of the stove. Each month's mix of signals will be varied and in combination unique, but together present a dynamically evolving pattern of behaviors that are reliably representative of a memory impairment ( 13 ). This is an oversimplified example of what a new world of digital biomarkers might look like.…”

Federated Adversarial Debiasing for Fair and Transferable Representations

“…Dataset Due to the mild symptoms and expansive cost of clinic diagnosis, early detection of MCI is a hard task. To address the challenge, MCI detection models is built on a MCI dataset, which is collected with Intelligent Systems for Assessing Aging Change (ISAAC), a longitudinal cohort study [18,20]. A total of 152 participants were enrolled beginning in 2017.…”

“…Though prior work has shown the effectiveness of machine learning methods in diagnosis prediction [18,25], the possibility of training such a model fairly in a distributed framework remains unknown. We assume the sensor data can be immediately trained locally and only the trained models are sent to the server.…”