Modes of Information Flow

Abstract

Information flow between components of a system takes many forms and is key to understanding the organization and functioning of large-scale, complex systems. We demonstrate three modalities of information flow from time series X to time series Y . Intrinsic information flow exists when the past of X is individually predictive of the present of Y , independent of Y 's past; this is most commonly considered information flow. Shared information flow exists when X's past is predictive of Y 's present in the same manner as Y 's past; this occurs due to synchronization or common driving, for example. Finally, synergistic information flow occurs when neither X's nor Y 's pasts are predictive of Y 's present on their own, but taken together they are. The two most broadly-employed information-theoretic methods of quantifying information flow-time-delayed mutual information and transfer entropy-are both sensitive to a pair of these modalities: time-delayed mutual information to both intrinsic and shared flow, and transfer entropy to both intrinsic and synergistic flow. To quantify each mode individually we introduce our cryptographic flow ansatz, positing that intrinsic flow is synonymous with secret key agreement between X and Y . Based on this, we employ an easily-computed secret-key-agreement bound-intrinsic mutual information-to quantify the three flow modalities in a variety of systems including asymmetric flows and financial markets.

Unique information via dependency constraints

“…3 as a more concrete example to draw out such differences. 4 Table I gives the measures' decomposition values. Interestingly, I min agrees with I proj , while I ccs agrees with I BROJA .…”

Inferring directional interactions in collective dynamics: a critique to intrinsic mutual information

“…Rephrasing James et al [19], transfer entropy is sensitive to both intrinsic dependencies between Z t and Y t+1 , as well as the dependencies induced by Y t . To filter the latter dependencies and precisely measure information flow, Sattari et al [21] proposed the use of intrinsic mutual information from Z to Y, defined as…”

“…Building on this key limitation, James et al [19] have recently detailed information flow within a pair of dynamical systems, distinguishing multiple, co-existing modalities of information flow that have been erroneously compounded in the literature. Among them, 'intrinsic information flow' pertains to the predictive power that the present of the source alone has on the target's future, independent of the target's present: this quantity is what is routinely referred to as information flow (but seldom precisely measured).…”

Transfer entropy dependent on distance among agents in quantifying leader-follower relationships

“…Using a simple binary model, it was demonstrated that the transfer entropy between two processes is not localized in the way that the expression of "transfer from one to another" [73], hence transfer entropy is not appropriate to characterize 'information flow' in some cases. However, in recent works [17,74], it is shown that transfer entropy can be decomposed into two distinct modes of information flow, namely intrinsic and synergistic. This intrinsic information from X to Y is designed to extract solely the information flow from X to Y, buried in transfer entropy.…”