Quantifying synergistic mutual information

Griffith, Virgil; Koch, Christof

doi:10.48550/arxiv.1205.4265

Cited by 17 publications

(59 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It remains to be seen whether the overall nature of these results would change if using an alternative redundancy measure to I min (e.g. [21], [22]).…”

Section: Discussionmentioning

confidence: 99%

“…I min measures redundancy as the minimum amount of information which can be found in any source A j . This has been criticized since it does not specifically require each source to hold the same information, as demonstrated with the "twobit copy problem" [21], [22], [24], which is the observation that I min ({A 1 , A 2 }; A 1 , A 2 ) = 1 bit for independent random bits A 1 , A 2 . This observation prompted the proposal of a new axiom for I ∩ [22]:…”

Section: B the I Min Measure For Redundancymentioning

confidence: 99%

“…(This is called the PI-diagram for four variables in [11], including the destination variable.) and Griffith and Koch [21]. We describe these briefly in Section VI-C, though focus on I min in our current study as the originally-presented concrete measure.…”

Section: B the I Min Measure For Redundancymentioning

confidence: 99%

“…There is the prospect that alternate measures satisfying the existing axioms for I ∩ may satisfy the axioms we have laid out above for localizing redundancy and information modification. Two candidates here [21], [22] were proposed to address the two-bit copy problem raised with I min .…”

Section: Prospects With Other Candidate Redundancy Measuresmentioning

confidence: 99%

“…Griffith and Koch propose to measure the redundancy by mapping the destination X to a surrogate X which preserves the information from each source A j to the surrogate, but minimizes the overall mutual information from the sources to the surrogate [21]. This method at first seems localizable (by simply localizing the MI between the sources and the surrogate), however as pointed out in [21] the mapping (i.e. PDFs) to produce the minimal MI is not unique.…”

Section: Prospects With Other Candidate Redundancy Measuresmentioning

confidence: 99%

See 4 more Smart Citations

Towards a synergy-based approach to measuring information modification

Lizier

Flecker

Williams

2013

2013 IEEE Symposium on Artificial Life (ALife)

105

View full text Add to dashboard Cite

Distributed computation in artificial life and complex systems is often described in terms of component operations on information: information storage, transfer and modification. Information modification remains poorly described however, with the popularly-understood examples of glider and particle collisions in cellular automata being only quantitatively identified to date using a heuristic (separable information) rather than a proper information-theoretic measure. We outline how a recentlyintroduced axiomatic framework for measuring information redundancy and synergy, called partial information decomposition, can be applied to a perspective of distributed computation in order to quantify component operations on information. Using this framework, we propose a new measure of information modification that captures the intuitive understanding of information modification events as those involving interactions between two or more information sources. We also consider how the local dynamics of information modification in space and time could be measured, and suggest a new axiom that redundancy measures would need to meet in order to make such local measurements. Finally, we evaluate the potential for existing redundancy measures to meet this localizability axiom.

show abstract

“…It remains to be seen whether the overall nature of these results would change if using an alternative redundancy measure to I min (e.g. [21], [22]).…”

Section: Discussionmentioning

confidence: 99%

Section: B the I Min Measure For Redundancymentioning

confidence: 99%

Section: B the I Min Measure For Redundancymentioning

confidence: 99%

Section: Prospects With Other Candidate Redundancy Measuresmentioning

confidence: 99%

Section: Prospects With Other Candidate Redundancy Measuresmentioning

confidence: 99%

See 3 more Smart Citations

Towards a synergy-based approach to measuring information modification

Lizier

Flecker

Williams

2013

2013 IEEE Symposium on Artificial Life (ALife)

105

View full text Add to dashboard Cite

show abstract

Bivariate measure of redundant information

2013

View full text Add to dashboard Cite

We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behaviour of this new measure for several examples, compare it to previous measures and apply it to the decomposition of transfer entropy.

show abstract

Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems

2015

View full text Add to dashboard Cite

To fully characterize the information that two source variables carry about a third target variable, one must decompose the total information into redundant, unique, and synergistic components, i.e., obtain a partial information decomposition (PID). However, Shannon's theory of information does not provide formulas to fully determine these quantities. Several recent studies have begun addressing this. Some possible definitions for PID quantities have been proposed and some analyses have been carried out on systems composed of discrete variables. Here we present an in-depth analysis of PIDs on Gaussian systems, both static and dynamical. We show that, for a broad class of Gaussian systems, previously proposed PID formulas imply that (i) redundancy reduces to the minimum information provided by either source variable and hence is independent of correlation between sources, and (ii) synergy is the extra information contributed by the weaker source when the stronger source is known and can either increase or decrease with correlation between sources. We find that Gaussian systems frequently exhibit net synergy, i.e., the information carried jointly by both sources is greater than the sum of information carried by each source individually. Drawing from several explicit examples, we discuss the implications of these findings for measures of information transfer and information-based measures of complexity, both generally and within a neuroscience setting. Importantly, by providing independent formulas for synergy and redundancy applicable to continuous time-series data, we provide an approach to characterizing and quantifying information sharing amongst complex system variables.

show abstract

Quantifying synergistic mutual information

Cited by 17 publications

References 0 publications

Towards a synergy-based approach to measuring information modification

Towards a synergy-based approach to measuring information modification

Bivariate measure of redundant information

Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems

Contact Info

Product

Resources

About