Optimization-based Causal Estimation from Heterogenous Environments

Abstract

This paper presents a new optimization approach to causal estimation. Given data that contains covariates and an outcome, which covariates are causes of the outcome, and what is the strength of the causality? In classical machine learning (ML), the goal of optimization is to maximize predictive accuracy. However, some covariates might exhibit a non-causal association to the outcome. Such spurious associations provide predictive power for classical ML, but they prevent us from causally interpreting the result. This paper proposes CoCo, an optimization algorithm that bridges the gap between pure prediction and causal inference. CoCo leverages the recently-proposed idea of environments, datasets of covariates/response where the causal relationships remain invariant but where the distribution of the covariates changes from environment to environment. Given datasets from multiple environments-and ones that exhibit sufficient heterogeneity-CoCo maximizes an objective for which the only solution is the causal solution. We describe the theoretical foundations of this approach and demonstrate its effectiveness on simulated and real datasets. Compared to classical ML and existing methods, CoCo provides more accurate estimates of the causal model.

Environment Invariant Linear Least Squares

“…We aim to estimate β * and S * using the n • |E| data {(x (e) i , y (e) i )} e∈E,i∈{1,...,n} . The model assumptions of multiple environments resemble and slightly relax the assumptions in this paper's predecessors, for example, Peters et al (2016); Rojas-Carulla et al (2018); Pfister et al (2021); Yin et al (2021).…”

“…Such a multiple-environment setting is common in many applications (Meinshausen et al, 2016;Čuklina et al, 2021). Hence there is a considerable literature proposing methods to estimate β * and S * , for example, Peters et al (2016); Rojas-Carulla et al (2018); Arjovsky et al (2019); Pfister et al (2021); Yin et al (2021); Rothenhäusler et al (2019Rothenhäusler et al ( , 2021. However, there are limited theoretical understandings on the methods proposed for the general model (1.2).…”

“…However, such procedures may collapse to conservative solutions, for example, S = ∅, and there is no finite-sample guarantee for the power of the test. There is also a considerable literature (Ghassami et al, 2017;Rothenhäusler et al, 2019Rothenhäusler et al, , 2021Pfister et al, 2021;Yin et al, 2021) proposing regression methods to estimate the causal parameter β * using the invariance principle, but these works either lack a finite-sample theoretical guarantee or need to impose strong restrictions on the linear model and the heterogeneity. The method we proposed can be applied to the causal estimation scenario and address the above concerns: our proposed estimator is the first one that is applicable in the general setup (1.4) with finite-sample theoretical guarantees, and it will not collapse to conservative solutions in the presence of independent important variables.…”

Invariant Feature Learning Based on Causal Inference from Heterogeneous Environments

“…It is crucial for the OOD problem to distinguish which features of the data are affected by the environment or exhibit spurious correlations with the target and which features are direct causes of the target [4]. In most out-of-distribution generalization problems, features acting as direct causes of the target variable maintain an invariant joint distribution with the target variable.…”

“…where x and y represent the observational data and labels, respectively; e represents the environment, which can be obtained from a heterogeneous environments dataset; and Φ c (•) : X → R c and Φ e (•) : X → R e are the feature extraction processes [1,4]. We refer to Φ c (x) and Φ e (x) as causal features and environmental features, respectively.…”