To Impute or not to Impute? Missing Data in Treatment Effect Estimation

Abstract

Missing data is a systemic problem in practical scenarios that causes noise and bias when estimating treatment effects. This makes treatment effect estimation from data with missingness a particularly tricky endeavour. A key reason for this is that standard assumptions on missingness are rendered insufficient due to the presence of an additional variable, treatment, besides the individual and the outcome. Having a treatment variable introduces additional complexity with respect to why some variables are missing that is not fully explored by previous work. In our work we identify a new missingness mechanism, which we term mixed confounded missingness (MCM), where some missingness determines treatment selection and other missingness is determined by treatment selection. Given MCM, we show that naively imputing all data leads to poor performing treatment effects models, as the act of imputation effectively removes information necessary to provide unbiased estimates. However, no imputation at all also leads to biased estimates, as missingness determined by treatment divides the population in distinct subpopulations, where estimates across these populations will be biased. Our solution is selective imputation, where we use insights from MCM to inform precisely which variables should be imputed and which should not. We empirically demonstrate how various learners benefit from selective imputation compared to other solutions for missing data.

Causal Deep Learning

“…Depending on which imputation strategy we use, we may introduce additional parametric assumptions into our pipeline. Furthermore, depending on which mechanism governs the missingness patterns, we may have to make additional structural assumptions before imputation can even begin [14,16,77]. Making such a structural assumptions, before learning a graph using Peters et al [73] would make our solution for the practitioner's problem invalid once again.…”

“…CDL can help define interaction effects of missingness and covariates to improve the accuracy of imputation methods. For example, CDL-based approaches allow deep learning to accurately impute missing values, respecting the causal interaction between missingness indicators and treatment selection [14][15][16]. CDL-based methods outperform existing imputation techniques in terms of both imputation accuracy and unbiased estimation from data with missing values.…”

Data Imputation for Clinical Trial Emulation: A Case Study on Impact of Intracranial Pressure Monitoring for Traumatic Brain Injury

“…However, few literature investigated these strategies in the estimation of medical treatment effect. Berrevoets et al found that selective imputation performed better than naively impute all data in causal context 11 , but it’s difficult to identify either missingness causing treatment or missingness caused by treatment pattern in practice. Thus our study aimed to compare various MI strategies with complete case analysis in treatment effect estimation systematically to evaluate whether imputing data works better and which imputation strategy should be adopted in RCT emulation using RWD.…”