Information-Theoretic Inference of Common Ancestors

Abstract

A directed acyclic graph (DAG) partially represents the conditional independence structure among observations of a system if the local Markov condition holds, that is if every variable is independent of its non-descendants given its parents. In general, there is a whole class of DAGs that represents a given set of conditional independence relations. We are interested in properties of this class that can be derived from observations of a subsystem only. To this end, we prove an information-theoretic inequality that allows for the inference of common ancestors of observed parts in any DAG representing some unknown larger system. More explicitly, we show that a large amount of dependence in terms of mutual information among the observations implies the existence of a common ancestor that distributes this information. Within the causal interpretation of DAGs, our result can be seen as a quantitative extension of Reichenbach's principle of common cause to more than two variables. Our conclusions are valid also for non-probabilistic observations, such as binary strings, since we state the proof for an axiomatized notion of "mutual information" that includes the stochastic as well as the algorithmic version.

Discriminating between causal structures in Bayesian Networks given partial observations

“…. , Y k and allows the following quantitative extension of Reichenbach's principle of common cause, proven in [12]. This extended common cause principle allows the discrimination between different causal models for a system by observation alone, even when Reichenbach's common cause principle would fail.…”

“…In our framework, this can be understood in the following way: If X and Y are part of a larger system, modeled by a dependency graph G and they are stochastically dependent, then their ancestral sets must be overlapping. Otherwise they would be d-separated by the empty set (which means X ⊥ ⊥ Y | ∅) and thus be independent [12].…”

“…We will now turn to a quantitative extension of the common cause principle, initially studied in [2] and later extended in [12]. Assume that we have a Bayesian network with variables X 1 , .…”

“…, X n can be observed. From these observations, it is possible to gain information about G, for example by Reichenbach's common cause principle: If two of the observed nodes are dependent, they must have a common ancestor in G. A quantitative version of this principle [2,12] allows to infer certain aspects of the causal structure using information theoretic quantities. We extend this line of work by deriving a tight upper bound on the quantity…”

“…Following [12], we ask which assertions about the structure of possible causal graphs can be made if we have no additional information beyond the joint probability distribution of the observed variables. Given a system consisting of observable quantities X 1 , .…”