Computationally tractable pairwise complexity profile

Bar-Yam, Yavni; Harmon, Dion; Bar‐Yam, Yaneer

doi:10.1002/cplx.21437

Cited by 23 publications

(33 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case we may choose the units of scale so that each component has scale equal to 1. This convention was used in previous work on the complexity profile [13][14][15][16][17]. However, it is in many cases necessary to represent the components of a system as having different intrinsic scales, reflecting their built-in size, multiplicity or redundancy.…”

Section: Scalementioning

confidence: 99%

“…Here we define the shared information, I A (x), of a dependency x in a system A. I generalizes the multivariate mutual information [13][14][15][16][17][30][31][32][33][34][35][36][37][38][39]] to an arbitrary information function H. We note that H and I characterize the same quantity-information-but are applied to different kinds of arguments: H is applied to subsets of components of A, while I is applied to dependencies.…”

Section: Information Quantity In Dependenciesmentioning

confidence: 99%

“…Statistical physics [24][25][26][27]-in particular the renormalization group of phase transitions [28,29]-provides a notion of scale in which individual components acting in concert can be considered equivalent to larger-scale units. Information theory provides the tool of multivariate mutual information: the information shared among an arbitrary number of variables (also called interaction information or co-information) [13][14][15][16][17][30][31][32][33][34][35][36][37][38][39]. These threads were combined in the complexity profile [13][14][15][16][17][18], a quantitative index of structure that characterizes the amount of information applying at a given scale or higher.…”

Section: Introductionmentioning

confidence: 99%

“…Information theory provides the tool of multivariate mutual information: the information shared among an arbitrary number of variables (also called interaction information or co-information) [13][14][15][16][17][30][31][32][33][34][35][36][37][38][39]. These threads were combined in the complexity profile [13][14][15][16][17][18], a quantitative index of structure that characterizes the amount of information applying at a given scale or higher. In the context of the complexity profile, the multivariate mutual information of a set of n variables is considered to have scale n. In this way, information is understood to have scale equal to the multiplicity (or redundancy) at which it arises-an idea which is also implicit in other works on multivariate mutual information [40,41].…”

Section: Introductionmentioning

confidence: 99%

“…Overcoming this limitation requires a multiscale approach to information theory [13][14][15][16][17][18]-one that identifies not only the amount of information in a given observable but also its scale, defined as the number or volume of components to which it applies. Information describing a star's position applies at a much larger scale than information describing a moon's position.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Multiscale Information Theory and the Marginal Utility of Information

Allen

Stacey

Bar‐Yam

2017

Entropy

Self Cite

View full text Add to dashboard Cite

Complex systems display behavior at a range of scales. Large-scale behaviors can emerge from the correlated or dependent behavior of individual small-scale components. To capture this observation in a rigorous and general way, we introduce a formalism for multiscale information theory. Dependent behavior among system components results in overlapping or shared information. A system's structure is revealed in the sharing of information across the system's dependencies, each of which has an associated scale. Counting information according to its scale yields the quantity of scale-weighted information, which is conserved when a system is reorganized. In the interest of flexibility we allow information to be quantified using any function that satisfies two basic axioms. Shannon information and vector space dimension are examples. We discuss two quantitative indices that summarize system structure: an existing index, the complexity profile, and a new index, the marginal utility of information. Using simple examples, we show how these indices capture the multiscale structure of complex systems in a quantitative way.

show abstract

Section: Scalementioning

confidence: 99%

Section: Information Quantity In Dependenciesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multiscale Information Theory and the Marginal Utility of Information

Allen

Stacey

Bar‐Yam

2017

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

On the eigenvalue and Shannon's entropy of finite length random sequences

Liu

Miao

et al. 2014

Complexity

View full text Add to dashboard Cite

Pseudorandom binary sequences play a significant role in many fields, such as spread spectrum communications, stochastic computation, and cryptography. The complexity measures of sequences and their relationship still remain an interesting open problem. In this article, we study on the eigenvalue of random sequences, deduce its theoretical expectation and variance of random sequences with length N, and establish the relationship between eigenvalue and Shannon's entropy. The results show that these two measures are consistent. Furthermore, the eigenvalue of random n-block sequences and its relation to Shannon's entropy are also been studied. V C 2014 Wiley Periodicals, Inc. Complexity 21: [154][155][156][157][158][159][160][161] 2015

show abstract

From big data to important information

Bar‐Yam

2016

Complexity

Self Cite

View full text Add to dashboard Cite

Advances in science are being sought in newly available opportunities to collect massive quantities of data about complex systems. While key advances are being made in detailed mapping of systems, how to relate this data to solving many of the challenges facing humanity is unclear. The questions we often wish to address require identifying the impact of interventions on the system and that impact is not apparent in the detailed data that is available. Here we review key concepts and motivate a general framework for building larger scale views of complex systems and for characterizing the importance of information in physical, biological and social systems. We provide examples of its application to evolutionary biology with relevance to ecology, biodiversity, pandemics, and human lifespan, and in the context of social systems with relevance to ethnic violence, global food prices, and stock market panic. Framing scientific inquiry as an effort to determine what is important and unimportant is a means for advancing our understanding and addressing many practical concerns, such as economic development or treating disease. I. OVERVIEWChanging disease to health and economic instability to growth are among the complex challenges we face today. How can we turn the massive quantities of data that are increasingly available towards addressing these pressing problems? The data provide abundant detail, but generally carry no labels for guidance about which pieces of information are important for determining successful interventions. The questions we need to address are about properties of complex systems-human physiology, the global economy. Addressing questions about such systems requires disentangling the intricate dependencies and multiple causes and effects of behaviors, and recognizing that behaviors range across scales from microscopic to macroscopic.Here we argue that the key to addressing these questions is to focus on the way behavior at different scales are related, and how dependencies within a system lead to the large scale patterns of behavior that can be characterized directly without mapping all of the intricate details. The approach builds upon an understanding of how to aggregate component behaviors to identify larger scale behaviors, an approach developed in the "renormalization group" study of phase transitions in physics, and generalized here to multiscale information theory. In this framework, information itself has scale and larger scale information is the most important information to know, with progressively finer scale information only of importance to provide detail when necessary. This analysis focuses attention on information characterizing how to affect the largest scale behaviors of the system. The method is a shortcut for the infeasible effort of mapping all of the causes and effects that extend from molecular to global scales of biological and social systems. Specific causes and effects that need to be studied are only a few compared to the many that underly the system behavior at all scales....

show abstract

Computationally tractable pairwise complexity profile

Abstract: Quantifying the complexity of systems consisting of many interacting parts has been an important challenge in

Cited by 23 publications

References 22 publications

Multiscale Information Theory and the Marginal Utility of Information

Multiscale Information Theory and the Marginal Utility of Information

On the eigenvalue and Shannon's entropy of finite length random sequences

From big data to important information

Contact Info

Product

Resources

About