Structural Equation Models and the Regression Bias for Measuring Correlates of Change

Abstract

ANCOVA and regression both exhibit a directional bias when measuring correlates of change. This bias confounds the comparison of changes between naturally occurring groups with large pretest differences (ANCOVA), or for identifying predictors of change when the predictor is correlated with pretest (regression). This bias is described in some detail. A computer simulation study is presented, which shows that properly identified structural equation models are not susceptible to this bias. Neither gain scores (posttest minus pretest) nor structural equation models exhibit the “regression bias.” Other factors, such as skewness, that may confound measurement of change are also discussed.

Health selection operating between classes and across employment statuses

“…By modelling changes of states between two consecutive waves rather than states themselves, the current study was partly able to adjust for pre-existing differences. Although modelling changes over time periods offers advantages over states-only design,39 in this model, health was defined as a ‘state’ variable. Accordingly, this modelling is not effective in dealing with a situation where people from a lower SEP were already in poor health outside the specification of the model in this study.…”

Analysis of covariance (ANCOVA) with difference scores

“…This raises the question of what analysis is appropriate for comparing changes in naturally occurring groups. If sufficient sample size and multiple measures are available, structural equation models of change (Raykov, 1992) are not susceptible to the ANCO-VA bias caused by measurement error (Cribbie and Jamieson, 2000). However, for the two-phase design with a single dependent variable, it is most appropriate to simply compare the posttest minus pretest difference scores using t-tests or ANOVA.…”

The impact of program structure on cortisol patterning in children attending out-of-home child care

“…The use of difference scores has received considerable criticism, such as their unreliability and regression to the mean. However, analysis of change scores is mathematically equivalent to a repeated-measures ANOVA and a paired t -test (Anderson et al, 1980), and a number of studies show that many concerns regarding change scores are unfounded (Cribbie & Jamieson, 2000; Edwards, 2001). The second approach (i.e., multilevel modeling of linear change in cortisol based on only two daily cortisol assessments) is controversial as it does not decompose the level-1 error in estimating the linear cortisol change and the level-2 variability that can be attributed to daily variations in cortisol diurnal patterns.…”