Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Abstract

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.

Causes of extreme events revealed by Rényi information transfer

“…In any case, the method presented here can be extended to multivariate settings and/or for including multidimensional variables (see the Supplementary Materials). Therefore the next step in this research is an implementation and testing of more effective estimation algorithms, such as those based on the k-nearest-neighbor search or kernel estimators, which have already been tested for the use in the RCMI/RTE analyses ( 53 , 54 ).…”

Quantification of Model Uncertainty Based on Variance and Entropy of Bernoulli Distribution

“…The limit for α  1 is Shannon entropy. Other properties of the Renyi entropy can be found, e.g., in [61].…”

A new transfer entropy method for measuring directed connectivity from complex-valued fMRI data

“…The most commonly used method for estimating real-valued transfer entropy is the histogram-based transfer entropy (HTE), which estimates the joint probability density function via a histogram-based function. Other transfer entropy algorithms were proposed to improve the accuracy of causal inference or noise robustness, including symbolic transfer entropy (STE) ( Li and Zhang, 2022 ), effective transfer entropy ( Behrendt et al, 2019 ; Caserini and Pagnottoni, 2022 ), Renyi transfer entropy ( Jizba et al, 2022 ; Zhang et al, 2023 ), and phase transfer entropy ( Wang and Chen, 2020 ; Gu et al, 2021 ).…”