Efficient Transfer Entropy Analysis of Non-Stationary Neural Time Series

Abstract

Information theory allows us to investigate information processing in neural systems in terms of information transfer, storage and modification. Especially the measure of information transfer, transfer entropy, has seen a dramatic surge of interest in neuroscience. Estimating transfer entropy from two processes requires the observation of multiple realizations of these processes to estimate associated probability density functions. To obtain these necessary observations, available estimators typically assume stationarity of processes to allow pooling of observations over time. This assumption however, is a major obstacle to the application of these estimators in neuroscience as observed processes are often non-stationary. As a solution, Gomez-Herrero and colleagues theoretically showed that the stationarity assumption may be avoided by estimating transfer entropy from an ensemble of realizations. Such an ensemble of realizations is often readily available in neuroscience experiments in the form of experimental trials. Thus, in this work we combine the ensemble method with a recently proposed transfer entropy estimator to make transfer entropy estimation applicable to non-stationary time series. We present an efficient implementation of the approach that is suitable for the increased computational demand of the ensemble method's practical application. In particular, we use a massively parallel implementation for a graphics processing unit to handle the computationally most heavy aspects of the ensemble method for transfer entropy estimation. We test the performance and robustness of our implementation on data from numerical simulations of stochastic processes. We also demonstrate the applicability of the ensemble method to magnetoencephalographic data. While we mainly evaluate the proposed method for neuroscience data, we expect it to be applicable in a variety of fields that are concerned with the analysis of information transfer in complex biological, social, and artificial systems.

Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics

“…While this eases considerably the estimation task, when the data distribution departs from Gaussianity the formulations in Equation (11) become approximate expressions of information dynamics, and the adopted estimator may miss dependence structures that originate from nonlinear dynamics. In such a case it is appropriate to resort to model-free computation methods, preferably those recently devised to tackle the difficult task of non-parametric entropy estimation in high dimensions [16,55,56] or for non-stationary data [57].…”

Quantifying Information Modification in Developing Neural Networks via Partial Information Decomposition

“…Attempts at formally defining information modification have presented a considerable challenge, however, in contrast to the well established measures of information transfer [2][3][4][5][6] and active information storage [7][8][9]. This is because identifying the "modified" information in the output of a processing element amounts to distinguishing it from the information from any input that survives the passage through the processor in unmodified form.…”

Advanced nonlinear approach to quantify directed interactions within EEG activity of children with temporal lobe epilepsy in their time course

“…Linear GC-based measures such as AR-based gPDC are time-variant and multivariate, and often also work in the presence of nonlinearity (for example see [35], comprehensive comparison of CCM and GC in the supplement of [13]). Because a time-varying investigation of TE is not trivial [12], TE was not adapted in our current study but should be investigated for these data in the future.…”

“…The transfer entropy (TE) [10,11] is a model-free measure of information transfer and is able to generalize GC to the detection of many different types of linear and nonlinear interactions in multivariate time series [9]. Time-variant investigations by means of TE are afflicted with the problem of data length needed for estimation; for the efficiency of TE in analyzing nonstationary data see [12].…”