Comparing Models of Change to Estimate the Mediated Effect in the Pretest–Posttest Control Group Design

Abstract

Models to assess mediation in the pretest-posttest control group design are understudied in the behavioral sciences even though it is the design of choice for evaluating experimental manipulations. The paper provides analytical comparisons of the four most commonly used models used to estimate the mediated effect in this design: Analysis of Covariance (ANCOVA), difference score, residualized change score, and cross-sectional model. Each of these models are fitted using a Latent Change Score specification and a simulation study assessed bias, Type I error, power, and confidence interval coverage of the four models. All but the ANCOVA model make stringent assumptions about the stability and cross-lagged relations of the mediator and outcome that may not be plausible in real-world applications. When these assumptions do not hold, Type I error and statistical power results suggest that only the ANCOVA model has good performance. The four models are applied to an empirical example.

An experiment on the impact of coaches’ and athlete leaders’ competence support on athletes’ motivation and performance

“…As a final step, we wanted to confirm the results provided by the path analysis, as difference scores have been criticized previously (see Henk and Castro‐Schilo for an historical overview). To do so, we implemented the suggestions of Valente and MacKinnon and assessed the mediation using analyses of covariance (ANCOVA) fitted with the latent change score (LCS) specification. In order to be able to fit our original model (Figure ) to the model suggested by Valente and MacKinnon (Figure ), we performed 10 separate ANCOVA's fitted with the LCS specification.…”

“…To do so, we implemented the suggestions of Valente and MacKinnon and assessed the mediation using analyses of covariance (ANCOVA) fitted with the latent change score (LCS) specification. In order to be able to fit our original model (Figure ) to the model suggested by Valente and MacKinnon (Figure ), we performed 10 separate ANCOVA's fitted with the LCS specification. These results displayed in Table corroborate our previous findings on the dual pathway and the impact of coaches and athlete leaders.…”

Residential green and blue space associated with better mental health: a pilot follow-up study in university students

“…Next, longitudinal mediation was tested using crosslagged panel mediation models, which are appropriate for only two measurement points (52). More specifically, we tested whether baseline NDVI and blue space predicted GHQ-12 at follow-up through mediator variables measured in May 2018.…”

Preventive parenting intervention during childhood and young black adults’ unhealthful behaviors: a randomized controlled trial

“…Contingent upon empirical support for the protective effect of prevention, we plan to take these findings one step further by examining the possibility that SAAF‐induced increases in supportive parenting mediated the prevention effects that emerged. To do this, we estimated a mediation model with latent difference scores, following the steps that Valente and MacKinnon () presented, for drug use among men who experienced high neighborhood SES disadvantage (top 40%, n = 91, 50 in control group and 41 in SAAF group), and for BMI among women who experienced such disadvantage (top 40%, n = 112, 61 in control group and 51 in SAAF group). The sample size of subgroup analyses was based on power analyses to detect the main effect of prevention status.…”