Bias in Before–After Studies: Narrative Overview for Anesthesiologists

Abstract

Before-after study designs are effective research tools and in some cases, have changed practice. These designs, however, are inherently susceptible to bias (ie, systematic errors) that are sometimes subtle but can invalidate their conclusions. This overview provides examples of before-after studies relevant to anesthesiologists to illustrate potential sources of bias, including selection/assignment, history, regression to the mean, test-retest, maturation, observer, retrospective, Hawthorne, instrumentation, attrition, and reporting/publication bias. Mitigating strategies include using a control group, blinding, matching before and after cohorts, minimizing the time lag between cohorts, using prospective data collection with consistent measuring/reporting criteria, time series data collection, and/or alternative study designs, when possible. Improved reporting with enforcement of the Enhancing Quality and Transparency of Health Research (EQUATOR) checklists will serve to increase transparency and aid in interpretation. By highlighting the potential types of bias and strategies to improve transparency and mitigate flaws, this overview aims to better equip anesthesiologists in designing and/or critically appraising before-after studies.

Perioperative goal-directed therapy: what’s the best study design to investigate its impact on patient outcome?

“…Instead of performing a randomized controlled trial, Lima et al [6] chose an uncontrolled before-and-after study design [14] (Fig. 1) comparing two periods, the one before (March to August 2013) and that after (March to August 2014) implementation of their treatment protocol.…”

“…Before-and-after studies are observational quasi-experimental effectiveness studies and compare the periods before and after the implementation of any new diagnostic or therapeutic strategy, e.g., a therapeutic intervention, treatment protocol, assessment tool, safety initiative, quality improvement project, guideline, or educational program [14]. Because they are simpler to perform, require fewer resources, and bear less ethical concerns than randomized controlled trials, before-and-after studies are frequently used in perioperative research [14], and have been performed to show that perioperative goal-directed therapy can improve patient outcome in high-risk abdominal surgery patients [15][16][17][18][19].…”

“…Because they are simpler to perform, require fewer resources, and bear less ethical concerns than randomized controlled trials, before-and-after studies are frequently used in perioperative research [14], and have been performed to show that perioperative goal-directed therapy can improve patient outcome in high-risk abdominal surgery patients [15][16][17][18][19].…”

“…Therefore, beforeand-after studies are supposed to enable the impact of an implemented treatment protocol on patient outcome to be assessed under "real-life conditions". To reflect real-life conditions, however, medical caregivers should be blinded to the fact that the implementation of a new treatment is accompanied by a clinical study [14]. This includes that no research coordinators or study nurses should be present during patient treatment, which may impair data quality in general and makes it particularly challenging to guarantee and systematically record the compliance to the protocol.…”

“…This includes that no research coordinators or study nurses should be present during patient treatment, which may impair data quality in general and makes it particularly challenging to guarantee and systematically record the compliance to the protocol. In addition, if possible, researchers assessing the outcome data should be blinded to the study period the patient data were recorded to avoid "observer bias" and should use the same outcome definitions for control and intervention group patients to avoid "measurement bias" [14]. Because the intervention group and the historical group are treated at different time points, before-and-after studies, by definition, are not randomized and thus prone to be confounded by any change in patient management that is not related to the study intervention but occurs before or during its implementation ("history bias") [14].…”