Computerized Cognitive Testing for Use in Clinical Trials: A Comparison of the Nih Toolbox and Cogstate C3 Batteries

Abstract

Background: As prevention trials for Alzheimer’s disease move into asymptomatic populations, identifying older individuals who manifest the earliest cognitive signs of Alzheimer’s disease is critical. Computerized cognitive testing has the potential to replace current gold standard paper and pencil measures and may be a more efficient means of assessing cognition. However, more empirical evidence about the comparability of novel computerized batteries to paper and pencil measures is required. Objectives: To determine whether two computerized IPad batteries, the NIH Toolbox Cognition Battery and Cogstate-C3, similarly predict subtle cognitive impairment identified using the Preclinical Alzheimer Cognitive Composite (PACC). Design, Setting, Participants: A pilot sample of 50 clinically normal older adults (Mage=68.5 years±7.6, 45% non-Caucasian) completed the PACC assessment, and the NIH Toolbox and Cogstate-C3 at research centers of Massachusetts General and Brigham and Women’s Hospitals. Participants made 3-4 in-clinic visits, receiving the PACC first, then the NIH Toolbox, and finally the Cogstate-C3. Measurements: Performance on the PACC was dichotomized by typical performance (>= 0.5SD), versus subtle cognitive impairment (<0.5SD). Composites for each computerized battery were created using principle components analysis, and compared with the PACC using non-parametric Spearman correlations. Logistic regression analyses were used to determine which composite was best able to classify subtle cognitive impairment from typical performance. Results: The NIH Toolbox formed one composite and exhibited the strongest within-battery alignment, while the Cogstate-C3 formed two distinct composites (Learning-Memory and Processing Speed-Attention). The NIH Toolbox and C3 Learning-Memory composites exhibited positive correlations with the PACC (ρ=0.49, p<0.001; ρ=0.58, p<0.001, respectively), but not the C3 Processing Speed-Attention composite, ρ=-0.18, p=0.22. The C3 Learning-Memory was the only composite that classified subtle cognitive impairment, and demonstrated the greatest sensitivity (62%) and specificity (81%) for that subtle cognitive impairment. Conclusions: Preliminary findings suggest that the NIH Toolbox has the advantage of showing the strongest overall clustering and alignment with standardized paper-and-pencil tasks. By contrast, Learning-Memory tasks within the Cogstate-C3 battery have the greatest potential to identify cross-sectional, subtle cognitive impairment as defined by the PACC.

A Neuropsychological Perspective on Defining Cognitive Impairment in the Clinical Study of Alzheimer’s Disease: Towards a More Continuous Approach

“…These features are particularly attractive for the detection of subtle, early cognitive changes associated with preclinical stage of AD, which may not otherwise be detected by the examiner. Numerous digital cognitive test batteries, such as the NIH Toolbox and Cambridge Neuropsychological Test Automated Battery (CANTAB) [57] as well as standalone tests, such as the (e.g., DCTclock [58]) and have been developed for face-to-face administration in clinical and research settings [59]. Unsupervised online neuropsychological testing such as the Cogstate Brief Battery has been used in several AD clinical trials.…”

Monthly At-Home Computerized Cognitive Testing to Detect Diminished Practice Effects in Preclinical Alzheimer's Disease

“…Computerized cognitive testing has the potential to capture changes in cognition earlier, by enabling standardized administration and data analyses allowing for remote, unsupervised, and more frequent assessments (e.g., monthly rather than yearly) in a feasible way (Gold et al, 2018 ; Koo and Vizer, 2019 ). Several computerized tests have been developed for use in remote, unsupervised settings, including the Computerized Cognitive Composite (C3) battery, which was designed to assess cognitive processes that rely on the medial temporal lobe (MTL) (Rentz et al, 2016 ; Buckley et al, 2017 ; Papp et al, 2021b ). The C3 comprises two well-validated episodic memory paradigms: the Face Name Associative Memory Exam (FNAME) (Rentz et al, 2011 ) and the Behavioral Pattern Separation Task—Object Version (BPSO) (Stark et al, 2013 ), and the Cogstate Brief Battery (CBB) (Maruff et al, 2009 ; Lim et al, 2012 ).…”

“…The C3 comprises two well-validated episodic memory paradigms: the Face Name Associative Memory Exam (FNAME) (Rentz et al, 2011 ) and the Behavioral Pattern Separation Task—Object Version (BPSO) (Stark et al, 2013 ), and the Cogstate Brief Battery (CBB) (Maruff et al, 2009 ; Lim et al, 2012 ). It was recently shown that unsupervised, at-home C3 testing on an iPad was feasible and could provide data that discriminated reliably between cognitively normal and impaired adults (Rentz et al, 2016 ; Buckley et al, 2017 ; Papp et al, 2021b ).…”

Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease

“…By allowing participants to complete study assessments using their own electronic device, data collection exponentially increases, improving clinical trial efficiency. Interest in using digital and remote assessments is growing and multiple studies have demonstrated the feasibility and validity of computerized assessments in supervised 4–6 and unsupervised settings 7,8 using personal devices 9,10 . Several of these assessments are designed for specific symptomatic groups (e.g., for detection of mild cognitive impairment [MCI]) 9 or broad populations 6 …”