Generalised Measures of Multivariate Information Content

Finn, Conor; Lizier, Joseph T.

doi:10.3390/e22020216

Cited by 21 publications

(29 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the tight relation between information dynamics and the VAR representation of Gaussian stochastic processes, future works can be envisaged to introduce ANNs for the estimation of measures of information dynamics different than the GC ( Faes et al, 2017b ; Finn & Lizier, 2020 ), computed even across multiple time scales ( Faes, Marinazzo & Stramaglia, 2017 ; Martins et al, 2020 ). Moreover, this new method will easily find application even in different contexts, such as the study of dynamic information flow between stock market indices ( Scagliarini et al, 2020 ), between different brain regions with Granger-based estimators ( Astolfi et al, 2007 ), for time series analysis in climatology ( Faes et al, 2017a ), or for the study of gene regulatory networks ( Davidson & Levin, 2005 ).…”

Section: Conclusion and Limitationsmentioning

confidence: 99%

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

Antonacci

Minati

Faes

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

One of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (VAR) model and on evaluating the information flow between two processes in terms of prediction error variances. In its most advanced setting, GC analysis is performed through a state-space (SS) representation of the VAR model that allows to compute both conditional and unconditional forms of GC by solving only one regression problem. While this problem is typically solved through Ordinary Least Square (OLS) estimation, a viable alternative is to use Artificial Neural Networks (ANNs) implemented in a simple structure with one input and one output layer and trained in a way such that the weights matrix corresponds to the matrix of VAR parameters. In this work, we introduce an ANN combined with SS models for the computation of GC. The ANN is trained through the Stochastic Gradient Descent L1 (SGD-L1) algorithm, and a cumulative penalty inspired from penalized regression is applied to the network weights to encourage sparsity. Simulating networks of coupled Gaussian systems, we show how the combination of ANNs and SGD-L1 allows to mitigate the strong reduction in accuracy of OLS identification in settings of low ratio between number of time series points and of VAR parameters. We also report how the performances in GC estimation are influenced by the number of iterations of gradient descent and by the learning rate used for training the ANN. We recommend using some specific combinations for these parameters to optimize the performance of GC estimation. Then, the performances of ANN and OLS are compared in terms of GC magnitude and statistical significance to highlight the potential of the new approach to reconstruct causal coupling strength and network topology even in challenging conditions of data paucity. The results highlight the importance of of a proper selection of regularization parameter which determines the degree of sparsity in the estimated network. Furthermore, we apply the two approaches to real data scenarios, to study the physiological network of brain and peripheral interactions in humans under different conditions of rest and mental stress, and the effects of the newly emerged concept of remote synchronization on the information exchanged in a ring of electronic oscillators. The results highlight how ANNs provide a mesoscopic description of the information exchanged in networks of multiple interacting physiological systems, preserving the most active causal interactions between cardiovascular, respiratory and brain systems. Moreover, ANNs can reconstruct the flow of directed information in a ring of oscillators whose statistical properties can be related to those of physiological networks.

show abstract

Section: Conclusion and Limitationsmentioning

confidence: 99%

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

Antonacci

Minati

Faes

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…net increases in redundancy from decreases in synergy. Future studies could perform more detailed analyses by employing partial information decomposition (PID) measures [74][75][76][77][78][79][80][81].…”

Section: Limitations and Future Workmentioning

confidence: 99%

High-order functional interactions in ageing explained via alterations in the connectome in a whole-brain model

Gatica

Rosas

Mediano

et al. 2021

Preprint

View full text Add to dashboard Cite

The human brain generates a rich repertoire of spatio-temporal activity patterns, which support a wide variety of motor and cognitive functions. These patterns of activity change with age in a multi-factorial manner. One of these factors is the variations in the brain's connectomics that occurs along the lifespan. However, the precise relationship between high-order functional interactions and connnectomics, as well as their variations with age are largely unknown, in part due to the absence of mechanistic models that can efficiently map brain connnectomics to functional connectivity in aging. To investigate this issue, we have built a neurobiologically-realistic whole-brain computational model using both anatomical and functional MRI data from 161 participants ranging from 10 to 80 years old. We show that the age differences in high-order functional interactions can be largely explained by variations in the connectome. Based on this finding, we propose a simple neurodegeneration model that is representative of normal physiological aging. As such, when applied to connectomes of young participant it reproduces the age-variations that occur in the high-order structure of the functional data. Overall, these results begin to disentangle the mechanisms by which structural changes in the connectome lead to functional differences in the ageing brain. Our model can also serve as a starting point for modelling more complex forms of pathological ageing or cognitive deficits.

show abstract

“…Note that the former measures do not set aside a special variable Y , meaning the information they quantify is conceptually different from the PID. This dichotomy is ever-present in the information decomposition literature, as illustrated by work on entropy decompositions [ 19 , 20 , 21 ] that do not specify Y . For this reason, we refer to the PID as a directed measure of MMI (as it is only defined with respect to a target variable), and the former approaches as undirected measures.…”

Section: MMI and Information Decompositionmentioning

confidence: 99%

Discovering Higher-Order Interactions Through Neural Information Decomposition

Reing

Steeg

Galstyan

2021

Entropy

View full text Add to dashboard Cite

If regularity in data takes the form of higher-order functions among groups of variables, models which are biased towards lower-order functions may easily mistake the data for noise. To distinguish whether this is the case, one must be able to quantify the contribution of different orders of dependence to the total information. Recent work in information theory attempts to do this through measures of multivariate mutual information (MMI) and information decomposition (ID). Despite substantial theoretical progress, practical issues related to tractability and learnability of higher-order functions are still largely unaddressed. In this work, we introduce a new approach to information decomposition—termed Neural Information Decomposition (NID)—which is both theoretically grounded, and can be efficiently estimated in practice using neural networks. We show on synthetic data that NID can learn to distinguish higher-order functions from noise, while many unsupervised probability models cannot. Additionally, we demonstrate the usefulness of this framework as a tool for exploring biological and artificial neural networks.

show abstract

Generalised Measures of Multivariate Information Content

Cited by 21 publications

References 77 publications

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

High-order functional interactions in ageing explained via alterations in the connectome in a whole-brain model

Discovering Higher-Order Interactions Through Neural Information Decomposition

Contact Info

Product

Resources

About