Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests

Abstract

Alzheimer's disease (AD) affects the memory and cognitive function of approximately 5.7 million Americans. Early detection subsequently allows for earlier treatment and improves outcomes. Currently, there exists a validated 30-min eye-tracking cognitive assessment (VPC-30) for predicting AD risk. However, a shorter assessment would improve user experience and improve scalability. Thus, the purposes were to (1) determine convergent validity between the 5-min web camera-based eyetracking task (VPC-5) and VPC-30, (2) examine the relationship between VPC-5 and gold-standard cognitive tests, and (3) determine the reliability and stability of VPC-5. This prospective study included two healthy cohorts: older adults (65+ years, n = 20) and younger adults (18-46 years, n = 24). Participants were tested on two separate occasions. Visit 1 included the Montreal Cognitive Assessment (Mo-CA), Digit Symbol Coding test (DSC), NIH Toolbox Cognitive Battery (NIHTB-CB), VPC-30, and VPC-5. Visit 2 occurred at least 14 days later; participants completed the VPC-5, DSC, NIHTB-CB, and dual-task walking assessments (DT). VPC-30 significantly correlated with VPC-5 at the first (p < .001) and second (p = .001) time points. VPC-5 and DSC (p < .01) and Pattern Comparison Processing Speed Test (PSPAC) (p = .01) were also correlated on day 1. Significant associations existed between VPC-5 and DSC (p < .001), Flanker Inhibitory Control Test (p = .05), PSPAC (p < .001), and Picture Sequence Memory Test (p = .02) during day 14 testing session. The test retest reliability of VPC-5 was significant (p < .001). VPC-5 displayed moderate convergent validity with the VPC-30 and gold-standard measures of cognition, while demonstrating strong stability, suggesting it is a valuable assessment for monitoring memory function.

Unsupervised high-frequency smartphone-based cognitive assessments are reliable, valid, and feasible in older adults at risk for Alzheimer disease.

“…Similarly, the ARC composite score was correlated with the global composite score (r = -0.53). Albeit smaller than anticipated, the correlations observed here were comparable, if not stronger, than correlations observed in other digital assessment studies including the Cambridge Neuropsychological Test Automated Battery (CANTAB; rs 0.14 to 0.39; Dorociac et al, 2021;Gills et al, 2019;Smith et al, 2013). Additionally, the individual ARC tasks and the ARC composite score showed comparable correlations with age as the conventional measures and global composite score.…”

Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease

“…Correlations between BRANCH versus traditional memory tests ranged from r = 0.302 to 0.472, which is comparable, if not higher, compared to correlations observed in other studies with digital assessments. 45 , 46 For example, correlations between Cambridge Neuropsychological Test Automated Battery (CANTAB) memory measures and traditional memory measures ranged from 0.14 to 0.39 47 whereas a digital version of a list‐learning test showed somewhat stronger correlations with the traditional task, exhibiting correlations ranging from .37 to .62. 48 However, it appears that the Signs Test, in its current configuration as a continuous learning paradigm, was not as strongly associated with gold standard cognitive measures; as such, we are considering modifying the paradigm in the future.…”

Cognitive management in a digital world

“…To date, there are a limited number of validated digital cognitive assessments and of these, most are not readily available to the public. Our group has recently validated a short (5-minute) digital cognitive measurement assessing declarative memory [ 5 – 7 ]. This assessment is positively correlated with well-established cognitive screening tools and reliably predicts current cognitive status [ 6 , 7 ].…”

“…Our group has recently validated a short (5-minute) digital cognitive measurement assessing declarative memory [ 5 – 7 ]. This assessment is positively correlated with well-established cognitive screening tools and reliably predicts current cognitive status [ 6 , 7 ]. Remote cognitive screenings provide health care professionals with an additional tool when assessing cognitive decline, are time efficient, and allow for physicians to give information on various treatments to combat further abnormal decline.…”