Causal Network Inference by Optimal Causation Entropy

Abstract

Abstract. The broad abundance of time series data, which is in sharp contrast to limited knowledge of the underlying network dynamic processes that produce such observations, calls for a rigorous and efficient method of causal network inference. Here we develop mathematical theory of causation entropy, an information-theoretic statistic designed for model-free causality inference. For stationary Markov processes, we prove that for a given node in the network, its causal parents forms the minimal set of nodes that maximizes causation entropy, a result we refer to as the optimal causation entropy principle. Furthermore, this principle guides us to develop computational and data efficient algorithms for causal network inference based on a two-step discovery and removal algorithm for time series data for a network-couple dynamical system. Validation in terms of analytical and numerical results for Gaussian processes on large random networks highlight that inference by our algorithm outperforms previous leading methods including conditioned Granger causality and transfer entropy. Interestingly, our numerical results suggest that the number of samples required for accurate inference depends strongly on network characteristics such as the density of links and information diffusion rate and not necessarily on the number of nodes.

Coupling functions: Universal insights into dynamical interaction mechanisms

“…In a theoretical treatment the coupling strength is clearly the scaling parameter of the coupling functions. There is great interest in being able to evaluate the coupling strength for which many effective methods have been designed (Mormann et al, 2000;Rosenblum and Pikovsky, 2001;Paluš and Stefanovska, 2003;Marwan et al, 2007;Bahraminasab et al, 2008;Staniek and Lehnertz, 2008;Chicharro and Andrzejak, 2009;Smirnov and Bezruchko, 2009;Jamšek, Paluš, and Stefanovska, 2010;Faes, Nollo, and Porta, 2011;Sun, Taylor, and Bollt, 2015). The dominant direction of influence, i.e., the direction of the stronger coupling, corresponds to the directionality of the interactions.…”

Introduction to Focus Issue: Causation inference and information flow in dynamical systems: Theory and applications

“…For these questions, one must cast a conditional variation of the above concepts. There exist conditional Granger causalities, 7,12,34 conditional transfer entropies, and state conditioned transfer entropies, 67 and a special variation leading to causation entropy 8,25,58,59 . Furthermore, if one wishes to uncover coupling structure, then we require an algorithm premised on these computations; for example, PC algorithm and momentary information-based causal discovery 46 which address inference of large-scale nonlinear causal networks in the presence of strong autocorrelation, or the optimal causation entropy (oCSE) approach 58,59 which is designed to uncover the network of direct information flow influences using (CSE) as the underlying influence measure.…”

Potential conditional mutual information: Estimators and properties

“…This was originally studied in the setting when the variables are jointly Gaussian and hence the dependence is linear (see [6] for the original treatment, and [7,8] for versions with latent variables). This problem was generalized to the setting with arbitrary probability distributions and temporal dependences in [9] and studied further in [10], for one-step markov chains in [11] and deterministic relationships in [12]. From these works, under some technical condition, we can assert that the following method is guaranteed to be consistent,…”