Psychometric Properties of the NIH Toolbox Cognition Battery in Healthy Older Adults: Reliability, Validity, and Agreement with Standard Neuropsychological Tests

Abstract

Objective: Few independent studies have examined the psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) in older adults, despite growing interest its use for clinical purposes. In this paper we report the test-retest reliability and construct validity of the NIHTB-CB, as well as its agreement or concordance with traditional neuropsychological tests of the same construct to determine whether tests could be used interchangeably. Methods: Sixty-one cognitively healthy adults ages 60-80 completed “gold standard” (GS) neuropsychological tests, NIHTB-CB, and brain MRI. Test-retest reliability, convergent/discriminant validity and agreement statistics were calculated using Pearson’s correlations, concordance correlation coefficients (CCC), and root mean square deviations. Results: Test-retest reliability was acceptable (CCC = .73 Fluid; CCC = .85 Crystallized). The NIHTB-CB Fluid Composite correlated significantly with cerebral volumes (r’s = ∣.35-.41∣), and both composites correlated highly with their respective GS composites (r’s = .58-.84), although this was more variable for individual tests. Absolute agreement was generally lower (CCC = .55 Fluid; CCC = .70 Crystallized) due to lower precision in Fluid scores and systematic overestimation of Crystallized Composite scores on the NIHTB-CB. Conclusions: These results support the reliability and validity of the NIHTB-CB in healthy older adults and suggest that the Fluid Composite tests are at least as sensitive as standard neuropsychological tests to medial temporal atrophy and ventricular expansion. However, the NIHTB-CB may generate different estimates of performance and should not be treated as interchangeable with established neuropsychological tests.

Reliability of the NIH toolbox cognitive battery in children and adolescents: a 3-year longitudinal examination

“…very short term). A more recent study assessed long-term stability (average 15.03 ± 3.11 months between retest) of NIH-TB uncorrected standardized scores in older adults (Scott et al, 2019), with results indicating a pattern more similar to our investigation. Namely, reliability indices were predominantly moderate in magnitude, though values ranged from 0.46 to 0.87 for individual tests (lower bound of 95% CI range: 0.14-0.77; Scott et al, 2019).…”

“…A more recent study assessed long-term stability (average 15.03 ± 3.11 months between retest) of NIH-TB uncorrected standardized scores in older adults (Scott et al, 2019), with results indicating a pattern more similar to our investigation. Namely, reliability indices were predominantly moderate in magnitude, though values ranged from 0.46 to 0.87 for individual tests (lower bound of 95% CI range: 0.14-0.77; Scott et al, 2019). It is possible that the long retest interval of the present study, as well as that of Scott et al, may have attenuated reliability estimates, and that in agreement with previous literature a shorter retest interval may have yielded better stability estimates.…”

Laptop-Administered NIH Toolbox and Cogstate Brief Battery in Community-Dwelling Black Adults: Unexpected Pattern of Cognitive Performance between MCI and Healthy Controls

“…Two computerized cognitive assessment batteries that are increasingly being used in clinical research are the Cogstate Brief Battery (CBB; Hammers et al, 2012;Hammers et al, 2011;Maruff et al, 2013) and the NIH Toolbox-Cognition Battery (NIHTB-CB; Heaton et al, 2014;Weintraub et al, 2013). These measures have demonstrated acceptable reliability and construct validity as compared to traditional paper-pencil methods (Casaletto et al, 2015;Heaton et al, 2014) but lack clear support as a replacement for goldstandard neuropsychological assessment (Scott, Sorrell, & Benitez, 2019). Although there has been significant effort put forth to develop representative normative samples for these measures, there continues to be a lack of knowledge around the performance on these two cognitive batteries by individuals from underrepresented populations.…”

“…For example, two of the largest studies on the psychometric properties of the CBB make no mention of the race/ethnicity of their samples (Maruff et al, 2013;Mielke et al, 2015), pointing to a lack of information regarding the use of these measures in ethnically diverse populations. With regard to the NIHTB-CB, the original normative data were unadjusted for demographic factors, with subsequent normative samples providing corrections for race/ethnicity (Casaletto et al, 2015); however, it has been found that the NIHTB-CB may overestimate such individuals' deficits (Scott et al, 2019) and more research is needed to validate the clinical utility of this measure. Overall, there remains a fundamental gap in our understanding of the utility of these computerized cognitive batteries for use in clinical research trials including Black adults.…”

“…We report a cross-sectional analysis that cannot evaluate the sensitivity of the cognitive assessment to decline and transition to disease; however, the between-group comparison and validation against the NACC-UDS identified critical directions for future research on computerized assessments. Computerrelated anxiety was included as covariate in all analyses, but there also remains a question as to whether computerized assessments may differentially bias performance across cognitive domains, especially assessments of crystallized functions (Scott et al, 2019). A more thorough understanding of how this may contribute to differences in sensitivity to detect MCI among Black adults is warranted.…”