Is the direction of greater Granger causal influence the same as the direction of information flow?

Abstract

Granger causality is an established statistical measure of the "causal influence" that one stochastic process X has on another process Y . Along with its more recent generalization -Directed Information -Granger Causality has been used extensively in neuroscience, and in complex interconnected systems in general, to infer statistical causal influences. More recently, many works compare the Granger causality metrics along forward and reverse links (from X to Y and from Y to X), and interpret the direction of greater causal influence as the "direction of information flow". In this paper, we question whether the direction yielded by comparing Granger Causality or Directed Information along forward and reverse links is always the same as the direction of information flow. We explore this question using two simple theoretical experiments, in which the true direction of information flow (the "ground truth") is known by design. The experiments are based on a communication system with a feedback channel, and employ a strategy inspired by the work of Schalkwijk and Kailath. We show that in these experiments, the direction of information flow can be opposite to the direction of greater Granger causal influence or Directed Information. We also provide information-theoretic intuition for why such counterexamples are not surprising, and why Granger causality-based information-flow inferences will only get more tenuous in larger networks. We conclude that one must not use comparison/difference of Granger causality to infer the direction of information flow.

Information Flow in Computational Systems

“…The absence of a systematic framework with well-defined assumptions inherently makes it very hard to draw sound inferences through the application of Granger Causality-based tools. A striking example of this is a recent result of ours that shows that Granger Causality-based methods can recover incorrect directions of information flow in the presence of feedback links [31], even in the absence of hidden nodes and measurement noise. The time-unrolled graph framework presented here has been specifically designed to address this issue, and present a clear understanding of information flow, even in the presence of feedback.…”

“…information "trickling" over time from one node to another-could still pose issues. Such information flow could go undetected, or worse, appear to be flowing in the opposite direction (i.e., from the receiver to the transmitter), as has been shown to occur in a different context [28,31], using different measures of flow. It is possible that Derived Information, in particular, is hard to infer in the presence of noise.…”

“…Approaches for statistically mapping functional connectivity often rely on variations of Granger Causality [12] and, more recently, Directed Information [14][15][16], which we here collectively dub "Granger Causality-based tools". These approaches lack a systematic framework that clearly lays down assumptions, however, and the interpretations drawn from their use have often been questioned [19,21,22,[28][29][30][31].…”

“…For instance, one fundamental question that has remained unanswered is: when can directed causal influence be interpreted as information flow? In previous work, we demonstrated that even under ideal measurement conditions, the direction of causal influence can be opposite to the direction of information flow in certain kinds of feedback communication networks [31]. Another fundamental issue with the use of directed causal influence measures has to do with interpreting what the influence is "about".…”

Measuring the Non-linear Directed Information Flow in Schizophrenia by Multivariate Transfer Entropy

“…Consequently, a signal activation in one area of the brain directly causes a change or signal, activation or depression, in another area (Mastrovito et al, 2018;Zhu et al, 2018). Effective connectivity in a domain of data-driven approaches such as Granger causality analysis (GCA) which performs poorly in non-linear context rely on its past to formulate linear causal interactions in the EEG signal (Venkatesh and Grover, 2016;. The GCA is initially formulated for linear models and later extended to nonlinear systems by applying to local linear models.…”