Are latent variable models preferable to composite score approaches when assessing risk factors of change? Evaluation of type-I error and statistical power in longitudinal cognitive studies

Abstract

As with many health constructs, cognition is difficult to measure accurately; it is assessed by multiple psychometric tests. Two approaches are commonly adopted to address this multivariate aspect in longitudinal analyses: the composite score approach summarizes the tests into a single outcome and subsequently analyzes its change; the multivariate approach relates the tests to the underlying cognitive level and simultaneously analyzes its change. We compared the quality of inference of these approaches in a simulation study based on three combinations of tests inspired by two population-based cohorts. In the absence of missing data and with relatively Gaussian psychometric tests, the composite score approach provided similar type-I error rates and statistical power as the multivariate latent process approach. In the more plausible scenario with departures from normality, transformations of each constituent test or of the composite score were required to avoid excess type-I error rates. When missing tests were more likely in cognitively impaired subjects, inference with the composite was not correct. In conclusion, composite scores can be used to assess risk factors for cognitive change provided they are correctly normalized, constituent tests are reliable and the amount of uninformative missing tests remains small. Otherwise, latent variable models are recommended.

Religious Orders Study and Rush Memory and Aging Project

“…The ROSMAP investigators are committed to timely data sharing of raw or processed data which can be found on our Resource Sharing Hub with relevant links to the AMP-AD Knowledge Portal [297, 298]. A sample of work done with ROSMAP data including as part of translational studies and consortia is provided to give the community a better sense of the range of work that could be done leveraging the resource [299–350].…”

Reaction to “Sufficient statistics and insufficient explanations”: Use your information

“…This nested random-effect structure introduces a dependence where the higher-level variance estimate is influenced by the lower-level variance estimate. Proust-Lima et al 3 did not detect biased Type-I errors of regression effects, but they also did not have this nested random effect structure in their multivariate latent variable model.…”

Assessing Change in Intervention Research: The Benefits of Composite Outcomes

“…By pooling different imperfect measures of an underlying ability, one is less prone to measurement error. Thus, composites can help tap shared variance and reduce task-specific variability and come with many desirable properties—they allow meaningful comparisons across studies and are often straightforward to interpret (Proust-Lima et al, 2019). For example, an intervention study that investigates the effect of a particular treatment on a composite depression score can be compared against another study using the same composite as well as against population norms measured in nonintervention settings.…”