Unified framework for information integration based on information geometry

Abstract

Assessment of causal influences is a ubiquitous and important subject across diverse research fields. Drawn from consciousness studies, integrated information is a measure that defines integration as the degree of causal influences among elements. Whereas pairwise causal influences between elements can be quantified with existing methods, quantifying multiple influences among many elements poses two major mathematical difficulties. First, overestimation occurs due to interdependence among influences if each influence is separately quantified in a part-based manner and then simply summed over. Second, it is difficult to isolate causal influences while avoiding noncausal confounding influences. To resolve these difficulties, we propose a theoretical framework based on information geometry for the quantification of multiple causal influences with a holistic approach. We derive a measure of integrated information, which is geometrically interpreted as the divergence between the actual probability distribution of a system and an approximated probability distribution where causal influences among elements are statistically disconnected. This framework provides intuitive geometric interpretations harmonizing various information theoretic measures in a unified manner, including mutual information, transfer entropy, stochastic interaction, and integrated information, each of which is characterized by how causal influences are disconnected. In addition to the mathematical assessment of consciousness, our framework should help to analyze causal relationships in complex systems in a complete and hierarchical manner.integrated information | mutual information | transfer entropy | information geometry | consciousness Q uantitative assessment of causal influences among elements in a complex system is a fundamental problem in many fields of science, including physics (1), economics (2), gene networks (3), social networks (4), ecosystems (5), and neuroscience (6). There have been many previous attempts to quantify causal influences between elements in stochastic systems. Information theory has played a pivotal role in these endeavors, leading to various measures, including predictive information (7), transfer entropy (8), and stochastic interaction (9). Drawn from consciousness studies involving measurement of integration of neural activity (10, 11), the mathematical concept of integrated information is also useful as a framework for analyzing causal relationships in complex systems with multiple elements.Recent research suggests that the brain loses the ability to integrate information when consciousness is lost during dreamless sleep (12), general anesthesia (13), or vegetative states (14), suggesting that quantifying integration of information can serve as a neurophysiological marker of consciousness (10,11,15). The integrated information theory (IIT) of consciousness (16,17) proposes a measure of integration called integrated information that quantifies multiple causal influences among elements of a system. Integrated inform...

Conscious Perception as Integrated Information Patterns in Human Electrocorticography

“…Entropy quantifies only the instantaneous uncertainty of the neural states, understandable as equal-time interactions (Oizumi et al, 2016b; Fig. 2 A ).…”

“What is it like to be a bat?”—a pathway to the answer from the integrated information theory

“…Applying these IIT concepts as they are to a real human brain, which is composed of 10 11 neurons and 10 14 synaptic connections, is currently impossible for practical purposes. Thus, we need some gross approximations for these concepts when we empirically test explanations and predictions from IIT (Barrett & Seth, ; Chang et al, ; Lee, Mashour, Kim, Noh, & Choi, ; Oizumi, Amari, et al, ; Oizumi, Tsuchiya, & Amari, ; Tegmark, ). With approximations, our research group has computed patterns of integrated information from real neural activities recorded in awake human patients while they reported what they see in each trial in several tasks (Haun et al, ; Figure ).…”

“…Applying these IIT concepts as they are to a real human brain, which is composed of 10 11 neurons and 10 14 synaptic connections, is currently impossible for practical purposes. Thus, we need some gross approximations for these concepts when we empirically test explanations and predictions from IIT (Barrett & Seth, 2011;Chang et al, 2012;Lee, Mashour, Kim, Noh, & Choi, 2009;Oizumi, Amari, et al, 2016a;Oizumi, Tsuchiya, & Amari, 2016b;Tegmark, 2016).…”

Bayesian Nonlinear Filtering via Information Geometric Optimization

“…The KL divergence is also called relative entropy as in information theory. As for the KL divergence, it is a good measure of difference with the desired mathematical properties [31]. Let q approximate the neighborhood p as q (y, θ) = p (y, θ + ∆θ), and the Taylor expansion gives an approximation of the KL divergence by…”

“…It is evident that the effect of nonlinear measurement is reflected by the first terms of the Fisher metric tensor. Equation (31) establishes the relationship between the nonlinear measurement and the metric tensor.…”