Evaluating a Key Instrumental Variable Assumption Using Randomization Tests

Abstract

Instrumental variable (IV) analyses are becoming common in health services research and epidemiology. Most IV analyses use naturally occurring instruments, such as distance to a hospital. In these analyses, investigators must assume the instrument is as-if randomly assigned. This assumption cannot be tested directly, but it can be falsified. Most IV falsification tests compare relative prevalence or bias in observed covariates between the instrument and exposure. These tests require investigators to make covariate-by-covariate judgments about the validity of the IV design. Often, only some covariates are well-balanced, making it unclear if as-if randomization can be assumed for the instrument. We propose an alternative falsification test that compares IV balance or bias to the balance or bias that would have been produced under randomization. A key advantage of our test is that it allows for global balance measures as well as easily interpretable graphical comparisons. Furthermore, our test does not rely on parametric assumptions and can be used to validly assess if the instrument is significantly closer to being as-if randomized than the exposure. We demonstrate our approach using (SPOT)light, a prospective cohort study in 48 National Health Service hospitals between 1 November 2010 and 31 December 2011. This study used bed availability in the intensive care unit as an instrument for admission to the intensive care unit.

Randomization Tests to Assess Covariate Balance When Designing and Analyzing Matched Datasets

“…Furthermore, our approach can be applied to settings beyond matching. For example, assumptions of random assignment have been used in regression discontinuity designs (Li et al, 2015;Cattaneo et al, 2015;Mattei & Mealli, 2016;Branson & Mealli, 2018) and instrumental variable approaches (Brookhart & Schneeweiss, 2007;Baiocchi et al, 2014;Branson & Keele, 2020). Our randChecks R package-used throughout this paper-can formally test effective random assignment of any binary indicator, such as binary treatments in regression discontinuity designs and binary instrumental variables.…”

Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies

“…In this way, the CPT procedure yields an exact p-value for testing the randomization assumption. Similar Fisherian-permutation-based strategy was also leveraged in Branson (2020) and Branson and Keele (2020) to deliver an exact test for the randomization assumption.…”

Profiling Compliers in Instrumental Variables Designs

“…In both cases, A3* can be probed using falsification tests. 10 In addition to assumptions A1, A2* and A3*, many IV analyses whose goal is to estimate the LATE invoke the following monotonicity assumption: In the IV design we consider, study units can be classified into four strata based on combinations of treatment exposure D and instrument Z. Table 1 contains the study population stratified by D and Z.…”

“…To improve the IV study design, an expanding literature in epidemiology has focused on developing guidelines, diagnostics, and falsification tests for evaluating these assumptions. [3][4][5][6][7][8][9][10] Here, we further contribute to this literature by developing new methods for profiling compliers.…”