Minimum Sample Size for Reliable Causal Inference Using Transfer Entropy

Abstract

Transfer Entropy has been applied to experimental datasets to unveil causality between variables. In particular, its application to non-stationary systems has posed a great challenge due to restrictions on the sample size. Here, we have investigated the minimum sample size that produces a reliable causal inference. The methodology has been applied to two prototypical models: the linear model autoregressive-moving average and the non-linear logistic map. The relationship between the Transfer Entropy value and the sample size has been systematically examined. Additionally, we have shown the dependence of the reliable sample size and the strength of coupling between the variables. Our methodology offers a realistic lower bound for the sample size to produce a reliable outcome.

A Tutorial for Information Theory in Neuroscience

“…However, small data sets will increase the likelihood of type 2 errors (i.e., failing to report a significant information theory result when one is actually present) from surrogate data significance testing and produce bias (see Bias in Entropy and Mutual Information ). The number of observations necessary to estimate a probability distribution has been explored to some extent in the literature (Ramos and Macau, 2017), but a great deal of attention has been paid to other methods to assess bias and estimate probability distributions (see Handling Continuous Data and Further Refinements ).…”

A Blueprint for Identifying Phenotypes and Drug Targets in Complex Disorders with Empirical Dynamics

“…The number of discrete states is determined by the length of the time series. 52 Two time series are then considered causally related if the information, or entropy in one discrete signal is reduced given information about the other discrete signal. The difference in entropy between the signal by itself and the signal conditioned on the second variable is the transfer entropy.…”

“…In this work, we examined three different methods of discretizing the state space, based on methods previously useful in biological time series 41 (uniformly spaced bins, bins whose size is inferred by kernel density estimation on a normal probability distribution fit to the data, and bins whose size is learned via the Darbellay-Vajda algorithm). 53 As the length of our time series informs the number of states used, 52 four were used in our analysis. We only considered a result to be positive if it was positive using all three discretization schemes.…”

Quantification of the Direct Solar Impact on Some Components of the Hydroclimatic System

“…The present investigation refers to the seasonal, unfiltered time series from 1948 to 2000. We therefore have a statistical volume of 53 values, sufficient for entropy transfer analysis [34].…”