Potential Bias of Instrumental Variable Analyses for Observational Comparative Effectiveness Research

Abstract

Instrumental variable analysis is an increasingly popular method in comparative effectiveness research (CER). In theory, the instrument controls for unobserved and observed patient characteristics that affect the outcome. However, the results of instrumental variable analyses in observational settings may be biased if the instrument and outcome are related through an unadjusted third variable: an "instrument-outcome confounder." The authors identified published CER studies that used instrumental variable analysis and searched the literature for potential confounders of the most common instrument-outcome pairs. Of the 187 studies identified, 114 used 1 or more of the 4 most common instrument categories: distance to facility, regional variation, facility variation, and physician variation. Of these, 65 used mortality as an outcome. Potential unadjusted instrument-outcome confounders were observed in all studies, including patient race, socioeconomic status, clinical risk factors, health status, and urban or rural residency; facility and procedure volume; and co-occurring treatments. Only 4 (6%) instrumental variable CER studies considered potential instrument-outcome confounders outside the study data. Many effect estimates may be biased by the failure to adjust for instrument-outcome confounding. The authors caution against overreliance on instrumental variable studies for CER.

Surgical clipping versus endovascular coiling for elderly patients presenting with subarachnoid hemorrhage

“…To overcome confounding (the non-random selection of patients for either treatment) due to covariates not captured in our data (such as the clinical condition of the patient upon presentation, aneurysm size and location) which cannot be addressed with traditional methods, we employed an IV analysis 10. This analysis uses the differences in practice patterns across regions to simulate the structure of a randomized trial in an observational setting.…”

Medicare expenditures for elderly patients undergoing surgical clipping or endovascular intervention for unruptured cerebral aneurysms

“…To overcome this confounding (the nonrandom selection of patients for either treatment) due to covariates not captured by Medicare analyses we employed an IV analysis. 27 This analysis uses the differences in practice patterns across regions to simulate the structure of a randomized trial in an observational setting. This advanced observational technique has been used before by clinical researchers to answer comparative effectiveness questions for different interventions.…”

Association of Hospitalization for Neurosurgical Operations in Magnet Hospitals With Mortality and Length of Stay

“…[17][18][19] A good instrument is only associated through the variable of interest with the outcome. 20 It is unlikely that the differential distance to the closest Magnet hospital is associated with the primary outcome in a different way, other than the choice of facility. We employed a 2-stage least squares (2SLS) method for the analysis.…”