Latent variable models for harmonization of test scores: A case study on memory

Abstract

Combining data from different studies has a long tradition within the scientific community. It requires that the same information is collected from each study to be able to pool individual data. When studies have implemented different methods or used different instruments (e.g., questionnaires) for measuring the same characteristics or constructs, the observed variables need to be harmonized in some way to obtain equivalent content information across studies. This paper formulates the main concepts for harmonizing test scores from different observational studies in terms of latent variable models. The concepts are formulated in terms of calibration, invariance, and exchangeability. Although similar ideas are present in measurement reliability and test equating, harmonization is different from measurement invariance and generalizes test equating. In addition, if a test score needs to be transformed to another test score, harmonization of variables is only possible under specific conditions. Observed test scores that connect all of the different studies, are necessary to be able to test the underlying assumptions of harmonization. The concepts of harmonization are illustrated on multiple memory test scores from three different Canadian studies.

Development of a common scale for measuring healthy ageing across the world: results from the ATHLOS consortium

“… 43 , 44 IRT also provides the means to scores in a universal reference, yielding the possibility of rescaling individual scores obtained in a group to an arbitrary reference scale of choice. 43 , 45 …”

Using imputation to provide harmonized longitudinal measures of cognition across AIBL and ADNI

“…This approach of standardizing or normalizing scores generates data distributions that are unit-free thereby allowing outcomes from theoretically similar tests in different studies to be combined. Such methods require that the same underlying information is collected by similar tests and the strategy is not suitable for harmonising data with discrete values 21 – 24 , non-normal distributions or ceiling/floor effects 18 . Second, latent variable models can be utilised to determine underlying latent factors from a set of multiple test scores 21 .…”

“…Such methods require that the same underlying information is collected by similar tests and the strategy is not suitable for harmonising data with discrete values 21 – 24 , non-normal distributions or ceiling/floor effects 18 . Second, latent variable models can be utilised to determine underlying latent factors from a set of multiple test scores 21 . These models require common ‘anchor’ variables and assume that the measured test scores provide the same underlying information across studies that is captured by the latent construct 21 .…”

“…Second, latent variable models can be utilised to determine underlying latent factors from a set of multiple test scores 21 . These models require common ‘anchor’ variables and assume that the measured test scores provide the same underlying information across studies that is captured by the latent construct 21 . Thirdly, imputation or maximum likelihood estimation can be used to impute values for a test not administered in one dataset that is administered in another.…”

“…The present study proposes a new approach which uses a machine learning algorithm to impute and harmonize cognitive test scores across studies, as an extension to existing imputation methods of harmonization. The underlying structures and relationships of the measured data 21 are exploited to impute cognitive test scores in one dataset where the test was not administered. This aligns that dataset to another dataset in which the test was measured, resulting in harmonized data across the two datasets.…”