Adversarial balancing-based representation learning for causal effect inference with observational data

Abstract

Learning causal effects from observational data greatly benefits a variety of domains such as health care, education, and sociology. For instance, one could estimate the impact of a new drug on specific individuals to assist clinical planning and improve the survival rate. In this paper, we focus on studying the problem of estimating the Conditional Average Treatment Effect (CATE) from observational data. The challenges for this problem are two-fold: on the one hand, we have to derive a causal estimator to estimate the causal quantity from observational data, in the presence of confounding bias; on the other hand, we have to deal with the identification of the CATE when the distributions of covariates over the treatment group units and the control units are imbalanced. To overcome these challenges, we propose a neural network framework called Adversarial Balancing-based representation learning for Causal Effect Inference (ABCEI), based on recent advances in representation learning. To ensure the identification of the CATE, ABCEI uses adversarial learning to balance the distributions of covariates in the treatment and the control group in the latent representation space, without any assumptions on the form of the treatment selection/assignment function. In addition, during the representation learning and balancing process, highly predictive information from the original covariate space might be lost. ABCEI can tackle this information loss problem by preserving useful information for predicting causal effects under the regularization of a mutual information estimator. The experimental results show that ABCEI is robust against treatment selection bias, and matches/outperforms the state-of-the-art approaches. Our experiments show promising results on several datasets, encompassing several health care (and other) domains.

Deep Learning of Potential Outcomes

“…• Reconstruction Loss. Some authors have proposed that reconstruction losses should be applied to representation layers to improve confidence in the invertability assumption (Du et al 2019;Zhang et al 2020). These losses simply minimize an L 2 norm between inputs and outputs to force the representation function to be able to reconstruct it's inputs, along with it's other tasks: L(X, X ) = ||X − X || 2…”

“…• Adversarial Loss. Rather than learn to predict the propensity score Du et al (2019), apply an adversarial gradient to force the representation layers to "unlearn" information about treatment assignment. This approach is also applied in Bica et al (2020a).…”

“…The use of the IPM loss in CFRNet (Shalit et al 2017) may also be viewed as an adversarial approach in that the representation layers are forced to maximize performance on two competing tasks: predicting outcomes and minimizing an IPM. Rather than using an IPM loss, other authors have trained propensity score estimators that send positive (rather than negative) gradients back to the representation layers (Atan et al 2018;Du et al 2019). This strategy explicitly decorrelates the covariate representations from the treatment (see Box 5).…”

Guest editorial: Special issue on mining for health

“…Next, we provide a summary of each paper and highlight the main contributions and relevance to the goals of the special issue. Du et al (2021) introduce a novel model, referred to as Adversarial Balancing-based representation learning for Causal Effect Inference. The paper focuses on observational data and highlights the selection bias problem by introducing a neural network encoder constrained by a mutual information estimator for minimizing the loss between representation and input covariates.…”

The Causal-Neural Connection: Expressiveness, Learnability, and Inference

“…Whenever identifiability is obtained through the backdoor criterion/conditional ignorability [55,Sec. 3.3.1], deep learning techniques can be leveraged to estimate such effects with impressive practical performance [60,49,45,30,65,66,34,61,15,25]. For effects that are identifiable through causal functionals that are not necessarily of the backdoor-form (e.g., frontdoor, napkin), other optimization/statistical techniques can be employed that enjoy properties such as double robustness and debiasedness [31,32,33].…”