Entropic Causal Inference

Abstract

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using Renyi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low H0 entropy (cardinality) in the true direction, it must have high H0 entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum H1 entropy (Shannon Entropy) is equivalent to the problem of finding minimum joint entropy given n marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for n=2 provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum Shannon entropy. Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models.

Waste generation, wealth and GHG emissions from the waste sector: Is Denmark on the path towards circular economy?

“…The causality relationship expressed in econometric modeling is now tested through the ANNs approach. According to Pearl (2009) and Kocaoglu et al (2017), we develop Feed-forward Neural Networks as a Structural Causal Models (SCMs), to verify how (in a predictive way) TMWG, GDPp, and Urban cause GGWS in Denmark.…”

Causal Structure Learning: A Combinatorial Perspective

“…Thus far, we have discussed identification strategies designed for continuous random variables. Similar results are achievable in the discrete case, e.g., by assuming that the exogenous noise terms have low entropy [93] or by assuming the existence of a (hidden) low cardinality representation of the cause variable that mediates its effects [24].…”

Causal Structure Learning: a Combinatorial Perspective

“…Similar results are achievable in the discrete case, e.g. by assuming that the exogenous noise terms have low entropy [93], or by assuming the existence of a (hidden) low cardinality representation of the cause variable that mediates its effects [24].…”