Evaluation of a speech-based AI system for early detection of Alzheimer’s disease remotely via smartphones

Abstract

BackgroundChanges in speech, language, and episodic and semantic memory are documented in Alzheimer’s disease (AD) years before routine diagnosis.AimsDevelop an Artificial Intelligence (AI) system detecting amyloid-confirmed prodromal and preclinical AD from speech collected remotely via participants’ smartphones.MethodA convenience sample of 133 participants with established amyloid beta and clinical diagnostic status (66 Aβ+, 67 Aβ-; 71 cognitively unimpaired (CU), 62 with mild cognitive impairment (MCI) or mild AD) completed clinical assessments for the AMYPRED study (NCT04828122). Participants completed optional remote assessments daily for 7-8 days, including the Automatic Story Recall Task (ASRT), a story recall paradigm with short and long variants, and immediate and delayed recall phases. Vector-based representations from each story source and transcribed retelling were produced using ParaBLEU, a paraphrase evaluation model. Representations were fed into logistic regression models trained with tournament leave-pair-out cross-validation analysis, predicting Aβ status and MCI/mild AD within the full sample and Aβ status in clinical diagnostic subsamples.FindingsAt least one full remote ASRT assessment was completed by 115 participants (mean age=69.6 (range 54-80); 63 female/52 male; 66 CU and 49 MCI/mild AD, 56 Aβ+ and 59 Aβ-). Using an average of 2.7 minutes of automatically transcribed speech from immediate recall of short stories, the AI system predicted MCI/mild AD in the full sample (AUC=0.85 +/- 0.08), and amyloid in MCI/mild AD (AUC=0.73 +/- 0.14) and CU subsamples (AUC=0.71 +/- 0.13). Amyloid classification within the full sample was no better than chance (AUC=0.57 +/- 0.11). Broadly similar results were reported for manually transcribed data, long ASRTs and delayed recall.InterpretationCombined with advanced AI language models, brief, remote speech-based testing offers simple, accessible and cost-effective screening for early stage AD.FundingNovoic.Research in contextEvidence before this studyRecent systematic reviews have examined the use of speech data to detect vocal and linguistic changes taking place in Alzheimer’s dementia. Most of this research has been completed in the DementiaBank cohort, where subjects are usually in the (more progressed) dementia stages and without biomarker confirmation of Alzheimer’s disease (AD). Whether speech assessment can be used in a biomarker-confirmed, early stage (preclinical and prodromal) AD population has not yet been tested. Most prior work has relied on extracting manually defined “features”, e.g. the noun rate, which has too low a predictive value to offer clinical utility in an early stage AD population. In recent years, audio- and text-based machine learning models have improved significantly and a few studies have used such models in the context of classifying AD dementia. These approaches could offer greater sensitivity but it remains to be seen how well they work in a biomarker-confirmed, early stage AD population. Most studies have relied on controlled research settings and on manually transcribing speech before analysis, both of which limit broader applicability and use in clinical practice.Added value of this studyThis study tests the feasibility of advanced speech analysis for clinical testing of early stage AD. We present the results from a cross-sectional sample in the UK examining the predictive ability of fully automated speech-based testing in biomarker-confirmed early stage Alzheimer’s disease. We use a novel artificial intelligence (AI) system, which delivers sensitive indicators of AD-at-risk or subtle cognitive impairment. The AI system differentiates amyloid beta positive and amyloid beta negative subjects, and subjects with mild cognitive impairment (MCI) or mild AD from cognitively healthy subjects. Importantly the system is fully remote and self-contained: participants’ own devices are used for test administration and speech capture. Transcription and analyses are automated, with limited signal loss. Overall the results support the real-world applicability of speech-based assessment to detect early stage Alzheimer’s disease. While a number of medical devices have recently been approved using image-based AI algorithms, the present research is the first to demonstrate the use case and promise of speech-based AI systems for clinical practice.Implications of all the available evidencePrior research has shown compelling evidence of speech- and language-based changes occurring in more progressed stages of Alzheimer’s disease. Our study builds on this early work to show the clinical utility and feasibility of speech-based AI systems for the detection of Alzheimer’s disease in its earliest stages. Our work, using advanced AI systems, shows sensitivity to a biomarker-confirmed early stage AD population. Speech data can be collected with self-administered assessments completed in a real world setting, and analysed automatically. With the first treatment for AD entering the market, there is an urgent need for scalable, affordable, convenient and accessible testing to screen at-risk subject candidates for biomarker assessment and early cognitive impairment. Sensitive speech-based biomarkers may help to fulfil this unmet need.

AI‐based assessments of speech and language impairments in dementia

“…Studies have shown that AI assessments, 154 in particular, those developed based on individuals' 155 textual 116,120,123,152,153,156 or vocal samples 13,86,87,135,136,144,151,157,158 could potentially be integrated into dementia care settings. 10,14,159 This can help clinicians identify linguistic and acoustic impairments associated with the initial stages of dementia, 160,161 MCI, 162,163 and AD.…”

Protocol for Rhapsody: a longitudinal observational study examining the feasibility of speech phenotyping for remote assessment of neurodegenerative and psychiatric disorders

“…Demonstration that a binary classifier performs better than the random baseline will be done via mapping AUC estimates to a p value via the Mann-Whitney U statistic. 88 Results will be contrasted with demographic comparisons (comprising sex, education and age) as in prior analyses, 48 to evaluate the contribution of demographic imbalances.…”

“…These studies also typically report moderate to high levels of adherence to remote assessment and good acceptability of this method of assessment. Furthermore, digital speech capture can help to enrich analyses with more advanced text similarity analyses48 and automated extraction of language features commonly evaluated during connected speech,49 and furthermore incorporate vocal and acoustic features, which are sensitive to clinical status 50–53…”

Impact of Acoustic Noise on Alzheimer's Disease Detection from Speech: Should You Let Baby Cry?

“…Currently, speech recording for AD-related research typically takes place in a quiet room with a guiding clinician. Given that smartphone technology is rapidly advancing, speech assessments using ML models trained on recordings obtained by smartphones offer a potentially simple-to-administer and inexpensive solution, scalable to the entire population, that can be performed anywhere, including the patient's home [8,9,10]. However, the problem of model robustness to acoustic noise becomes increasingly important when deploying ML models in real world [11].…”