Quantifying causal influences

Abstract

Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution. Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1145 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

When the Map Is Better Than the Territory

“…Despite being interesting and useful metrics, these either fail to accurately measure causation, or are heuristics [8]. One difficultly is that information theory metrics, such as the mutual information, are usually thought to concern statistical correlations and are calculated over some observed distribution.…”

Part mutual information for quantifying direct associations in networks

“…However, recently, the causal strength C X→Y (X;YjZ) (see SI Appendix, Property S9) for quantifying causal strength was proposed by Janzing et al (18). The causal strength (CS) can be used to quantify causal influences among variables in a network.…”

“…In other words, strong dependency between X and Z (or between Y and Z) makes the conditional dependence of X and Y almost invisible when measuring CMI(X;YjZ) only (18). However, PMI(X;YjZ) can measure correctly for this case because the partial independence makes the conditional dependence of X and Y visible again by replacing p(xjz) p(yjz) with p*(xjz) p*(yjz), where p*(xjz) [or p*(yjz)] implicitly includes the association information between X and Y, different from p(xjz) [or p(yjz)].…”

“…probability p(x,y,z) ≠ 0], then CMI between X and Y given Z is always zero no matter whether X and Y are really dependent or not (18). Clearly, CMI underestimates the dependency because of this falsenegative shortcoming, in stark contrast to MI, which generally overestimates the dependency due to its false-positive drawback (16,19).…”

Information Geometry on Complexity and Stochastic Interaction

“…In order to overcome these limitations of the (conditional) mutual information, in a series of papers [35][36][37][38][39] we have proposed the use of information theory in combination with Pearl's theory of causation [40]. Our approach has been discussed in [41] where a variant of our notion of node exclusion, introduced in [36], has been utilized for an alternative definition. This definition, however, is restricted to direct causal effects and does not capture, in contrast to [35], mediated causal effects.…”