Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease

Papp, Kathryn V.; Samaroo, Aubryn; Chou, Hsiang‐Chin Lori; Buckley, Rachel F.; Schneider, Olivia R; Hsieh, Stephanie; Soberanes, Daniel; Quiroz, Yakeel T.; Properzi, Michael J.; Schultz, Aaron P.; García‐Magariño, Iván; Marshall, Gad A.; Burke, Jane G; Kumar, Raya; Snyder, Noah P.; Johnson, Keith A.; Rentz, Dorene M.; Sperling, Reisa A.; Amariglio, Rebecca E.

doi:10.1002/dad2.12243

Cited by 27 publications

(36 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ASRT tests correlated moderately with a well-established test of verbal episodic memory, cognitive composites and clinician reported outcomes, indicating acceptable convergent validity, and with results comparable to, or better than, other studies of computerised or unsupervised remote assessments (38–40). Correlations with the Logical Memory test and clinician reported outcomes were in the moderate range, with lower correlation coefficients linked to test invariance due to ceiling or floor-level performance on these assessments in cognitively unimpaired individuals.…”

Section: Discussionsupporting

confidence: 58%

Validation of a remote and fully automated story recall task to assess for early cognitive impairment in older adults: a longitudinal case-control observational study

Skirrow¹,

Meszaros²,

Meepegama³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

INTRODUCTION: Longitudinal data is key to identifying cognitive decline and treatment response in Alzheimer's disease (AD). METHODS: The Automatic Story Recall Task (ASRT) is a novel, fully automated test that can be self-administered remotely. In this longitudinal case-control observational study, 151 participants (mean age: 69.99 (range 54-82), 73 mild cognitive impairment/mild AD and 78 cognitively unimpaired) completed parallel ASRT assessments on their smart devices over 7-8 days. Responses were automatically transcribed and scored using text similarity metrics. RESULTS: Participants reported good task usability. Adherence to optional daily assessment was moderate. Parallel forms correlation coefficients between ASRTs were moderate-high. ASRTs correlated moderately with established tests of episodic memory and global cognitive function. Poorer performance was observed in participants with MCI/Mild AD. DISCUSSION: Unsupervised ASRT assessment is feasible in older and cognitively impaired people. This automated task shows good parallel forms reliability and convergent validity with established cognitive tests.

show abstract

Section: Discussionsupporting

confidence: 58%

Validation of a remote and fully automated story recall task to assess for early cognitive impairment in older adults: a longitudinal case-control observational study

Skirrow¹,

Meszaros²,

Meepegama³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…ASRT tests correlated moderately with a well-established test of verbal episodic memory, cognitive composites, and clinician-reported outcomes, indicating acceptable convergent validity, and with results comparable with, or better than, other studies of computerized or unsupervised remote assessments [ 41 - 43 ]. Correlations with LM and clinician-reported outcomes were in the moderate range, with lower correlation coefficients linked to test invariance owing to ceiling- or floor-level performance on these traditional clinical and neuropsychological assessments in CU individuals.…”

Section: Discussionsupporting

confidence: 56%

Validation of a Remote and Fully Automated Story Recall Task to Assess for Early Cognitive Impairment in Older Adults: Longitudinal Case-Control Observational Study

Skirrow¹,

Meszaros²,

Meepegama³

et al. 2022

JMIR Aging

View full text Add to dashboard Cite

Background Story recall is a simple and sensitive cognitive test that is commonly used to measure changes in episodic memory function in early Alzheimer disease (AD). Recent advances in digital technology and natural language processing methods make this test a candidate for automated administration and scoring. Multiple parallel test stimuli are required for higher-frequency disease monitoring. Objective This study aims to develop and validate a remote and fully automated story recall task, suitable for longitudinal assessment, in a population of older adults with and without mild cognitive impairment (MCI) or mild AD. Methods The “Amyloid Prediction in Early Stage Alzheimer’s disease” (AMYPRED) studies recruited participants in the United Kingdom (AMYPRED-UK: NCT04828122) and the United States (AMYPRED-US: NCT04928976). Participants were asked to complete optional daily self-administered assessments remotely on their smart devices over 7 to 8 days. Assessments included immediate and delayed recall of 3 stories from the Automatic Story Recall Task (ASRT), a test with multiple parallel stimuli (18 short stories and 18 long stories) balanced for key linguistic and discourse metrics. Verbal responses were recorded and securely transferred from participants’ personal devices and automatically transcribed and scored using text similarity metrics between the source text and retelling to derive a generalized match score. Group differences in adherence and task performance were examined using logistic and linear mixed models, respectively. Correlational analysis examined parallel-forms reliability of ASRTs and convergent validity with cognitive tests (Logical Memory Test and Preclinical Alzheimer’s Cognitive Composite with semantic processing). Acceptability and usability data were obtained using a remotely administered questionnaire. Results Of the 200 participants recruited in the AMYPRED studies, 151 (75.5%)—78 cognitively unimpaired (CU) and 73 MCI or mild AD—engaged in optional remote assessments. Adherence to daily assessment was moderate and did not decline over time but was higher in CU participants (ASRTs were completed each day by 73/106, 68.9% participants with MCI or mild AD and 78/94, 83% CU participants). Participants reported favorable task usability: infrequent technical problems, easy use of the app, and a broad interest in the tasks. Task performance improved modestly across the week and was better for immediate recall. The generalized match scores were lower in participants with MCI or mild AD (Cohen d=1.54). Parallel-forms reliability of ASRT stories was moderate to strong for immediate recall (mean rho 0.73, range 0.56-0.88) and delayed recall (mean rho=0.73, range=0.54-0.86). The ASRTs showed moderate convergent validity with established cognitive tests. Conclusions The unsupervised, self-administered ASRT task is sensitive to cognitive impairments in MCI and mild AD. The task showed good usability, high parallel-forms reliability, and high convergent validity with established cognitive tests. Remote, low-cost, low-burden, and automatically scored speech assessments could support diagnostic screening, health care, and treatment monitoring.

show abstract

“…In addition, they were related to tests of A cognitive measure often used in the early stages of AD is the PACC, and earlier studies have already compared unsupervised cognitive assessments against the PACC in similar samples. For example, a composite measure of the Boston Remote Assessment for Neurocognitive Health (BRANCH) (19) was strongly correlated with the PACC5 (r = 0.62) in cognitively normal participants. Another recent study (34) used a composite of similar but not identical tasks as used in this study and demonstrated a strong correlation to PACC (r = 0.51) in cognitively normal individuals without cognitive complaints.…”

Section: Discussionmentioning

confidence: 99%

Unsupervised mobile app-based cognitive testing in a population-based study of older adults born 1944

Öhman

Berron

Papp

et al. 2022

Front. Digit. Health

Self Cite

View full text Add to dashboard Cite

BackgroundMobile app-based tools have the potential to yield rapid, cost-effective, and sensitive measures for detecting dementia-related cognitive impairment in clinical and research settings. At the same time, there is a substantial need to validate these tools in real-life settings. The primary aim of this study was thus to evaluate the feasibility, validity, and reliability of mobile app-based tasks for assessing cognitive function in a population-based sample of older adults.MethodA total of 172 non-demented (Clinical Dementia Rating 0 and 0.5) older participants (aged 76–77) completed two mobile app-based memory tasks—the Mnemonic Discrimination Task for Objects and Scenes (MDT-OS) and the long-term (24 h) delayed Object-In-Room Recall Task (ORR-LDR). To determine the validity of the tasks for measuring relevant cognitive functions in this population, we assessed relationships with conventional cognitive tests. In addition, psychometric properties, including test-retest reliability, and the participants’ self-rated experience with mobile app-based cognitive tasks were assessed.ResultMDT-OS and ORR-LDR were weakly-to-moderately correlated with the Preclinical Alzheimer's Cognitive Composite (PACC5) (r = 0.3–0.44, p < .001) and with several other measures of episodic memory, processing speed, and executive function. Test-retest reliability was poor–to-moderate for one single session but improved to moderate–to-good when using the average of two sessions. We observed no significant floor or ceiling effects nor effects of education or gender on task performance. Contextual factors such as distractions and screen size did not significantly affect task performance. Most participants deemed the tasks interesting, but many rated them as highly challenging. While several participants reported distractions during tasks, most could concentrate well. However, there were difficulties in completing delayed recall tasks on time in this unsupervised and remote setting.ConclusionOur study proves the feasibility of mobile app-based cognitive assessments in a community sample of older adults, demonstrating its validity in relation to conventional cognitive measures and its reliability for repeated measurements over time. To further strengthen study adherence, future studies should implement additional measures to improve task completion on time.

show abstract

Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease

Cited by 27 publications

References 55 publications

Validation of a remote and fully automated story recall task to assess for early cognitive impairment in older adults: a longitudinal case-control observational study

Validation of a remote and fully automated story recall task to assess for early cognitive impairment in older adults: a longitudinal case-control observational study

Validation of a Remote and Fully Automated Story Recall Task to Assess for Early Cognitive Impairment in Older Adults: Longitudinal Case-Control Observational Study

Unsupervised mobile app-based cognitive testing in a population-based study of older adults born 1944

Contact Info

Product

Resources

About