Extending Transfer Entropy Improves Identification of Effective Connectivity in a Spiking Cortical Network Model

Abstract

Transfer entropy (TE) is an information-theoretic measure which has received recent attention in neuroscience for its potential to identify effective connectivity between neurons. Calculating TE for large ensembles of spiking neurons is computationally intensive, and has caused most investigators to probe neural interactions at only a single time delay and at a message length of only a single time bin. This is problematic, as synaptic delays between cortical neurons, for example, range from one to tens of milliseconds. In addition, neurons produce bursts of spikes spanning multiple time bins. To address these issues, here we introduce a free software package that allows TE to be measured at multiple delays and message lengths. To assess performance, we applied these extensions of TE to a spiking cortical network model (Izhikevich, 2006) with known connectivity and a range of synaptic delays. For comparison, we also investigated single-delay TE, at a message length of one bin (D1TE), and cross-correlation (CC) methods. We found that D1TE could identify 36% of true connections when evaluated at a false positive rate of 1%. For extended versions of TE, this dramatically improved to 73% of true connections. In addition, the connections correctly identified by extended versions of TE accounted for 85% of the total synaptic weight in the network. Cross correlation methods generally performed more poorly than extended TE, but were useful when data length was short. A computational performance analysis demonstrated that the algorithm for extended TE, when used on currently available desktop computers, could extract effective connectivity from 1 hr recordings containing 200 neurons in ∼5 min. We conclude that extending TE to multiple delays and message lengths improves its ability to assess effective connectivity between spiking neurons. These extensions to TE soon could become practical tools for experimentalists who record hundreds of spiking neurons.

Efficient computation and statistical assessment of transfer entropy

“…First, we want to get insight into an important aspect of all empirical studies on entropies: how to accurately build histograms, thus frequencies, as estimators for the probabilities in Equation (4). To this end, we follow the rationale of Hahs and Pethel [15] for the anticipatory system of Equation (9).…”

“…Clearly, the Z -scores for the independent system saturate at a negative, thus insignificant level, while the coupled system (full lines) show significant statistical power that continues to improve with increasing number of data points. The observations depend quantitatively on the number of bins of the histograms; however, the qualitative assessment is the same for number of bins ∈ [4,8,16].…”

“…Still, there might exist substantial overhead (e.g., I/O) that renders any parallelization attempt futile. 4 After initial distribution of the original data set to each thread. 5 The speedup is defined as the ratio of wall-time for a parallel program in comparison to a single-core, sequential version.…”

“…Besides the conceptual advantages of the TE, current approaches of using TE [4,5] face two major problems which we address in this work: (a) computation of the TE can be done directly using simple arrays for the data, but only inefficiently so, and, (b) while we can use the asymmetry TE(X → Y) = TE(Y → X) to guess on the direction of information flow, the TE itself does not allow for a statistical assessment of the significance of such flows and their respective directions.…”

Estimation of Transfer Entropy between Discrete and Continuous Random Processes

“…Firstly introduced by Schreiber [1], TE has been proposed as an effective measure of causality in industry [2], in order to detect where was a disturbance in the industrial process. TE has also been proposed as a powerful mean to detect neural connections in neuroscience [3]. Another application of TE in neuroscience has been to detect reliably the cerebral hemisphere containing epileptic focus.…”