Analyses of ‘change scores’ do not estimate causal effects in observational data

Abstract

Background In longitudinal data, it is common to create ‘change scores’ by subtracting measurements taken at baseline from those taken at follow-up, and then to analyse the resulting ‘change’ as the outcome variable. In observational data, this approach can produce misleading causal-effect estimates. The present article uses directed acyclic graphs (DAGs) and simple simulations to provide an accessible explanation for why change scores do not estimate causal effects in observational data. Methods Data were simulated to match three general scenarios in which the outcome variable at baseline was a (i) ‘competing exposure’ (i.e. a cause of the outcome that is neither caused by nor causes the exposure), (ii) confounder or (iii) mediator for the total causal effect of the exposure variable at baseline on the outcome variable at follow-up. Regression coefficients were compared between change-score analyses and the appropriate estimator(s) for the total and/or direct causal effect(s). Results Change-score analyses do not provide meaningful causal-effect estimates unless the baseline outcome variable is a ‘competing exposure’ for the effect of the exposure on the outcome at follow-up. Where the baseline outcome is a confounder or mediator, change-score analyses evaluate obscure estimands, which may diverge substantially in magnitude and direction from the total and direct causal effects. Conclusion Future observational studies that seek causal-effect estimates should avoid analysing change scores and adopt alternative analytical strategies.

Mental health in relation to changes in sleep, exercise, alcohol and diet during the COVID-19 pandemic: examination of four UK cohort studies

“…Effect estimates show the difference in risk for each outcome comparing those with the highest compared with least mental health symptoms. Because interpretation of within-person change scores can be problematic (Tennant, Arnold, Ellison, & Gilthorpe, 2021 ), main analyses examined associations between mental health and health behaviours at each timepoint; however, change score analyses are provided as additional analyses. Regression analyses were carried out by cohort and results were meta-analysed to formally assess heterogeneity using the I 2 statistic and obtain pooled estimates of association.…”

Does Duration of Pain at Baseline Influence Longer-term Clinical Outcomes of Low Back Pain Patients Managed on an Evidence-Based Pathway?

“…The baseline difference in EQ-5D VAS for the 12-month follow-up data set, while statistically significant was small and not clinically relevant. Because any association between our proposed predictor of response (duration of pain) and baseline pain is important in the context of casual inference, 29,30 we used an equivalence testing approach to verify the lack of association in our data. 31 The largest mean difference in pain ratings (0-10 scale) between pain duration groups <3 months and ≥12 months was found to be 0.3 units, equating to a standardised mean difference of 0.15.…”

“…It has been shown that an association between the proposed baseline predictor of response and the baseline value used in the calculation of change can bias inferences. 30 We found no association between the duration, and degree of pain at baseline (Supplementary Tables B, http://links.lww.com/BRS/B657 and C, http://links.lww.com/BRS/B657), and both baseline duration and status were entered into our statistical modal in an attempt to quantify independently adjusted influences. Nevertheless, we highlight the fact that our study was not a randomised controlled trial and so robust inferences about causality cannot be made.…”

Incorporating Baseline Outcome Data in Individual Participant Data Meta-Analysis of Non-randomized Studies

“…Among the published IPD-MA of NRSs in which continuous outcomes were assessed at baseline and follow-up (61% published since 2018), the change score was the most common statistical method, followed by ANCOVA—an unexpected finding because Cochrane recommends using ANCOVA to incorporate baseline outcome data in meta-analysis ( 45 ). A recent published paper by Tennant et al also recommends not to use change score in studies that aim to estimate a causal-effect because their results are not meaningful unless the baseline exposure and baseline outcome are independent from each other, which is extremely unlikely in non-randomized studies ( 46 ). However, Tennant et al also highlighted that adjustment for the baseline outcome, such as in ANCOVA, should not be made when the baseline outcome plausibly occurs after the exposure.…”