Synergy and Redundancy in Dual Decompositions of Mutual Information Gain and Information Loss

Secret sharing is a cryptographic discipline in which the goal is to distribute information about a secret over a set of participants in such a way that only specific authorized combinations of participants together can reconstruct the secret. Thus, secret sharing schemes are systems of variables in which it is very clearly specified which subsets have information about the secret. As such, they provide perfect model systems for information decompositions. However, following this intuition too far leads to an information decomposition with negative partial information terms, which are difficult to interpret. One possible explanation is that the partial information lattice proposed by Williams and Beer is incomplete and has to be extended to incorporate terms corresponding to higher-order redundancy. These results put bounds on information decompositions that follow the partial information framework, and they hint at where the partial information lattice needs to be improved.

Section: Remarkmentioning

confidence: 99%

Secret Sharing and Shared Information

Rauh

2017

“…[10] has been questioned as it can lead to unintuitive results [22], and thus many attempts have been devoted to finding alternative measures [22,[25][26][27][28][29][30] compatibly with an extended number of axioms, such as the identity axiom proposed in [22]. Other work has studied in more detail the lattice structure that underpins the PID, indicating the duality between information gain and information loss lattices [12]. Even though there is no consensus on how to build partial information decompositions in systems with more than two sources, for trivariate systems the measures of redundancy, synergy and unique information defined in Ref.…”

Section: Preliminaries and State Of The Artmentioning

confidence: 99%

“…For example, two of the variables, A and B, may carry either redundant or synergistic information about a third variable C [7][8][9], but considering the value of the mutual information I((A, B) : C) alone is not enough to distinguish these qualitatively different information-carrying modes. To achieve this finer level of understanding, recent theoretical efforts have focused on decomposing the mutual information between two subsets of variables into more specific information components (see e.g., [6,[10][11][12]). Nonetheless, a complete framework for the information-theoretic analysis of multivariate systems is still lacking.…”

Section: Introductionmentioning

confidence: 99%

Invariant Components of Synergy, Redundancy, and Unique Information among Three Variables

Pica

Piasini

Chicharro

et al. 2017

Self Cite

Abstract:In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to describe the relationships among all PID lattices. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system's entropy, hence it characterizes completely all the interdependencies in the system.

“…Information is understood to have a scale equal to the multiplicity (or redundancy) at which it arises, and so the analysis shows how these indices capture the multi-scale structure of complex systems. The work of Chicharro and Panzeri [4] also deals with the redundant aspects of information: it extends the framework of mutual information decomposition, based on the construction of information gain lattices, separating the information into the unique, redundant, and synergy components. In doing so, the work proposes a new construction of information gain and loss lattices.…”

mentioning

confidence: 99%

“…The issue begins with three papers which deal with the foundational aspects of information processing in complex systems [3][4][5]. The study of Allen et al [3] describes two quantitative indices that summarize the structure of a complex system: (i) its complexity profile, based on the multivariate mutual information at a given scale or higher, and (ii) the marginal utility of information, characterizing the extent to which a system can be described using limited amounts of information.…”

mentioning

confidence: 99%

Complexity, Criticality and Computation

Prokopenko

2017

What makes a system "complex"? Is it merely the number of components it integrates, a nonlinear nature of the dependencies and feedbacks among its parts, or an unpredictable behavior it exhibits over time? The term "complexity" was initially applied generically to express the lack of predictability, reflecting on the self-organization of a synergistic macroscopic behavior out of interactions between the constituent microscopic parts, and the emergence of global patterns. Without a doubt, by now the concept has acquired a fairly definitive meaning, describing a distinct field of research and education and a new approach to science and engineering. There are abundant examples showing that the enterprise of Complex Systems, having achieved a substantial level of maturity, reaches back into our everyday lives, revealing patterns of complexity that should be considered without employing a reductionist logic [1].Similarly, the idea of criticality was originally motivated by studies of various crises and disruptive events, as well as sensitivities to initial conditions, but over time has developed into a precise field: critical dynamics. Research into critical dynamics is typically focused on the behavior of dynamical spatiotemporal systems during phase transitions where scale invariance prevails and symmetries break. Crucially, such behavior can be understood in terms of the control and order parameters. For instance, a second-order phase transition in a ferromagnetic system, separating two qualitatively different behaviors, can be reached by controlling the temperature parameter: the "disordered" and isotropic (symmetric) high-temperature phase is characterized by the absence of net magnetization, while the "ordered" and anisotropic (less symmetric) low-temperature phase can be described by an order parameter, the net magnetization vector defining a preferred direction in space. Critical phenomena have become associated with the physics of critical points, such as fractal behavior, the divergence of the correlation length, power-law divergences (e.g., the divergence of the magnetic susceptibility in the ferromagnetic phase transition), universality of relevant critical exponents, and so on. Now, these precise theoretical notions begin to reconnect with their motivating applied studies of crisis modeling, forecasting, and response. There is a growing awareness that complexity is strongly related to criticality, and many examples of self-organizing complex systems can be found in applications managing complexity specifically at critical regimes.A similar loop originating in practical studies, maturing to an exact science with precise but narrow definitions, and then reaching back to applied scenarios, can be seen in the realm of distributed computation. These days, complex systems can be viewed as distributed information-processing systems, in the domains ranging from systems biology and artificial life to computational neuroscience, digital circuitry, and transport networks [2]. Consciousness emerging from neuronal ac...