Construct identification in the neuropsychological battery: What are we measuring?

Abstract

Objective: Major obstacles to data harmonization in neuropsychology include lack of consensus about what constructs and tests are most important and invariant across healthy and clinical populations. This study addressed these challenges using data from the National Neuropsychology Network (NNN). Method: Data were obtained from 5,000 NNN participants and Pearson standardization samples. Analyses included variables from four instruments: Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV); Wechsler Memory Scale, 4th Edition (WMS-IV); California Verbal Learning Test, 3rd Edition (CVLT3); and Delis-Kaplan Executive Function System (D-KEFS). We used confirmatory factor analysis to evaluate models suggested by prior work and examined fit statistics and measurement invariance across samples. We examined relations of factor scores to demographic and clinical characteristics. Results: For each instrument, we identified four first-order and one second-order factor. Optimal models in patients generally paralleled the best-fitting models in the standardization samples, including task-specific factors. Analysis of the NNN data prompted specification of a Recognition-Familiarity factor on the WMS-IV and an Inhibition-Switching factor on the D-KEFS. Analyses showed strong to strict factorial invariance across samples with expected differences in factor means and variances. The Recognition-Familiarity factor correlated with age more strongly in NNN than in the standardization sample. Conclusions: Factor models derived from healthy This document is copyrighted by the American Psychological Association or one of its allied publishers.This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Robert M. Bilder https://orcid.org/0000-0001-5085-7852 Robert M. Bilder has received consulting fees and/or honoraria over the last 12 months from Atai Life Sciences, the Institute of Digital Media and Child Development, and VeraSci. Dustin Wahlstrom and Louis Charles Vannier are salaried employees of Pearson Clinical Assessment, which publishes the Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV); Wechsler Memory Scale, 4th Edition (WMS-IV); California Verbal Learning Test, 3rd Edition (CVLT3); and Delis-Kaplan Executive Function System (D-KEFS).

Computerized adaptive test strategies for the matrix reasoning subtest of the Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV)

“…It remains unclear how much time is necessary to arrive at accurate estimates of true scores for these subtests; this could be a valuable topic for future research. Our prior confirmatory factor analyses further showed that WAIS-IV index scores, including the Perceptual Reasoning Index to which MR contributes, might be well estimated with only 2 subtests per index (Bilder et al, 2022).…”

“…Factor analytic work has shown that the Matrix Reasoning subtest loads together with other measures (Visual Puzzles, Block Design) to form the WAIS-IV Perceptual Reasoning Index (Wechsler et al, 2008;Wechsler, 2008a). We recently used confirmatory factor analysis to demonstrate that the WAIS-IV factor structure shows strong measurement invariance across a heterogeneous patient group from the NNN and the original healthy standardization sample (Bilder et al, 2022).…”

“…IRT can further specify the precision of measurement at different levels of the construct, which is often particularly important in the assessment of individuals with very high or very low levels of ability, and critical to assure that measurement properties are comparable within individuals when change has taken place due to disease or interventions. These methods are further important to assure that our tests measure the same constructs across groups that differ by sex, race, ethnicity, cultural backgrounds, and clinical conditions (Bilder et al, 2022).…”

Heart Rate Variability and Cognition: A Narrative Systematic Review of Longitudinal Studies

“…There was heterogeneity in the cognitive tests used across studies. This is likely due to the fact that, in the neuropsychological literature, a large variety of cognitive tests are available and there is no consensus on whether some are more robust than others [ 81 , 91 , 164 , 165 ]. In order to ensure better comparability between studies, it is thus important that research groups employ shared protocols for cognitive assessment.…”

Harmonization of neuropsychological and other clinical endpoints: Pitfalls and possibilities.

“…• In an effort to identify evidence-based and optimally efficient measurements of neuropsychological constructs that are valid across healthy and clinical populations, Bilder et al (2022) employ advanced psychometric methods utilized in the National Neuropsychology Network to identify the most salient cognitive constructs from flexible test batteries that are invariant across heterogenous groups.…”