Revealing the Dynamics of Neural Information Processing with Multivariate Information Decomposition

Abstract

The varied cognitive abilities and rich adaptive behaviors enabled by the animal nervous system are often described in terms of information processing. This framing raises the issue of how biological neural circuits actually process information, and some of the most fundamental outstanding questions in neuroscience center on understanding the mechanisms of neural information processing. Classical information theory has long been understood to be a natural framework within which information processing can be understood, and recent advances in the field of multivariate information theory offer new insights into the structure of computation in complex systems. In this review, we provide an introduction to the conceptual and practical issues associated with using multivariate information theory to analyze information processing in neural circuits, as well as discussing recent empirical work in this vein. Specifically, we provide an accessible introduction to the partial information decomposition (PID) framework. PID reveals redundant, unique, and synergistic modes by which neurons integrate information from multiple sources. We focus particularly on the synergistic mode, which quantifies the “higher-order” information carried in the patterns of multiple inputs and is not reducible to input from any single source. Recent work in a variety of model systems has revealed that synergistic dynamics are ubiquitous in neural circuitry and show reliable structure–function relationships, emerging disproportionately in neuronal rich clubs, downstream of recurrent connectivity, and in the convergence of correlated activity. We draw on the existing literature on higher-order information dynamics in neuronal networks to illustrate the insights that have been gained by taking an information decomposition perspective on neural activity. Finally, we briefly discuss future promising directions for information decomposition approaches to neuroscience, such as work on behaving animals, multi-target generalizations of PID, and time-resolved local analyses.

Information-processing dynamics in neural networks of macaque cerebral cortex reflect cognitive state and behavior

“…Note that in order to reduce the influence of behavior-dependent firing rate changes, the AIS and the m T E measures were significance-tested with conservatively estimated surrogate data (shuffling spikes within 25-ms windows and recomputing the corresponding measure; for details, SI Appendix , Materials and Methods ). In addition, all values ( AIS , m T E , and synergy) were normalized by dividing by the Shannon entropy of the receiver neuron, following ( 27 ), which provides a control for variable firing rates.…”

“…Unique information is the part of a target neuron’s activity that can only be expressed by the spiking of an individual source neuron. While the existence of synergy in biological neural networks is extremely well documented ( 27 ), evidence that behavioral-related neural processes can be captured with the computation of synergy is still lacking.…”

“…Previous work using information dynamics to study recorded neuronal networks found that the degree of information modification changes during development ( 21 ), and in the same developmental windows, particular patterns of information transfer are “locked-in” ( 23 ). Furthermore, the capacity for information modification is heterogeneously distributed across neurons of the network and concentrated in high-degree, rich-club neurons ( 24 – 27 ). Information transfer ( 28 ) has been applied to a variety of neural recordings (see ref.…”

Partial entropy decomposition reveals higher-order structures in human brain activity

“…BOLD itself is also fundamentally a proxy measure of brain activity based on oxygenated blood flow and not a direct measure of neural activity. Applying this work to electrophysciological data (M/EEG, which can be discretized in principled ways to enable information-theoretic analysis [60]), and naturally discrete spiking neural data [61], will help deepen our understanding of how higher-order interactions contribute to cognition and behavior. The applicability of the PED to multiple scales of analysis highlights one of the foundational strengths of the approach (and information-theoretic frameworks more broadly): being based on the fundamental logic of inferences under conditions of uncertainty, the PED can be applied to a large number of complex systems (beyond just the brain), or to multiple scales within a single system to provide a detailed, and holistic picture of the system's structure.…”

Untangling Synergistic Effects of Intersecting Social Identities with Partial Information Decomposition

“…Crucially, information theory is well-equipped to handle the problem of decomposing multivariate relationships in data into synergistic (intersectional) and redundant components using a framework known as partial information decomposition (PID) [ 30 , 31 ] (see Section 2.2 for details), and has been applied in a variety of fields, including interpretable machine learning [ 32 ], medical imaging [ 33 ], biological neural networks [ 34 , 35 ], ecology [ 36 ], evolution [ 37 ], as well as to philosophical questions such as the nature of “emergence” [ 38 , 39 , 40 ] and consciousness [ 41 ]. This interdisciplinary group of results suggests that synergistic relationships “greater than the sum of their parts” are ubiquitous in both natural and human-made systems, so it is natural to hypothesize that they may also exist in social systems.…”