We present an approach, framed in information theory, to assess nonlinear causality between the subsystems of a whole stochastic or deterministic dynamical system. The approach follows a sequential procedure for nonuniform embedding of multivariate time series, whereby embedding vectors are built progressively on the basis of a minimization criterion applied to the entropy of the present state of the system conditioned to its past states. A corrected conditional entropy estimator compensating for the biasing effect of single points in the quantized hyperspace is used to guarantee the existence of a minimum entropy rate at which to terminate the procedure. The causal coupling is detected according to the Granger notion of predictability improvement, and is quantified in terms of information transfer. We apply the approach to simulations of deterministic and stochastic systems, showing its superiority over standard uniform embedding. Effects of quantization, data length, and noise contamination are investigated. As practical applications, we consider the assessment of cardiovascular regulatory mechanisms from the analysis of heart period, arterial pressure, and respiration time series, and the investigation of the information flow across brain areas from multichannel scalp electroencephalographic recordings.

A challenge for physiologists and neuroscientists is to map information transfer between components of the systems that they study at different scales, in order to derive important knowledge on structure and function from the analysis of the recorded dynamics. The components of physiological networks often interact in a nonlinear way and through mechanisms which are in general not completely known. It is then safer that the method of choice for analyzing these interactions does not rely on any model or assumption on the nature of the data and their interactions. Transfer entropy has emerged as a powerful tool to quantify directed dynamical interactions. In this paper we compare different approaches to evaluate transfer entropy, some of them already proposed, some novel, and present their implementation in a freeware MATLAB toolbox. Applications to simulated and real data are presented.

Entropy measures are widely applied to quantify the complexity of dynamical systems in diverse fields. However, the practical application of entropy methods is challenging, due to the variety of entropy measures and estimators and the complexity of real-world time series, including nonstationarities and long-range correlations (LRC). We conduct a systematic study on the performance, bias, and limitations of three basic measures (entropy, conditional entropy, information storage) and three traditionally used estimators (linear, kernel, nearest neighbor). We investigate the dependence of entropy measures on estimator- and process-specific parameters, and we show the effects of three types of nonstationarities due to artifacts (trends, spikes, local variance change) in simulations of stochastic autoregressive processes. We also analyze the impact of LRC on the theoretical and estimated values of entropy measures. Finally, we apply entropy methods on heart rate variability data from subjects in different physiological states and clinical conditions. We find that entropy measures can only differentiate changes of specific types in cardiac dynamics and that appropriate preprocessing is vital for correct estimation and interpretation. Demonstrating the limitations of entropy methods and shedding light on how to mitigate bias and provide correct interpretations of results, this work can serve as a comprehensive reference for the application of entropy methods and the evaluation of existing studies.

Abstract:In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an alternative decomposition evidences the so-called cross entropy (CE) and conditional SE (cSE), quantifying the cross information and internal information of the target system, respectively. This study presents a thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical structure of coupled dynamic processes. First, we investigate the theoretical properties of these measures, providing the conditions for their existence and assessing the meaning of the information theoretic quantity that each of them reflects. Then, we present an approach for the exact computation of information dynamics based on the linear Gaussian approximation, and exploit this approach to characterize the behavior of SE, TE, CE and cSE in benchmark systems with known dynamics. Finally, we exploit these measures to study cardiorespiratory dynamics measured from healthy subjects during head-up tilt and paced breathing protocols. Our main result is that the combined evaluation of the measures of information dynamics allows to infer the causal effects associated with the observed OPEN ACCESSEntropy 2015, 17 278 dynamics and to interpret the alteration of these effects with changing experimental conditions.

In cardiovascular variability analysis, the significance of the coupling between two time series is commonly assessed by setting a threshold level in the coherence function. While traditionally used statistical tests consider only the parameters of the adopted estimator, the required zero-coherence level may be affected by some features of the observed series. In this study, three procedures, based on the generation of surrogate series sharing given properties with the original but being structurally uncoupled, were considered: independent identically distributed (IID), Fourier transform (FT), and autoregressive (AR). IID surrogates maintained the distribution of the original series, while FT and AR surrogates preserved the power spectrum. The ability of the three methods to define the threshold for zero coherence was validated and compared by computer simulations reproducing typical cardiovascular interactions. While the IID threshold depended only on record length and design parameters of the coherence estimator, FT and AR thresholds were frequency-dependent with peaks corresponding to the local maxima of the estimated coherence. FT and AR surrogates were able to compensate spurious coherence peaks due to equal-frequency but independent oscillations in the two series. The benefit of frequency-dependent thresholds was evident for short series with narrow-band oscillations. Thus, surrogates preserving the power spectrum of the original series are recommended to avoid false coupling detections in the presence of oscillations occurring at nearby frequencies but produced by different mechanisms, as may frequently happen in cardiovascular and cardiorespiratory regulation.

The proposed approach evaluates complexity of the cardiovascular control and causality among cardiovascular regulatory mechanisms from spontaneous variability of heart period (HP), systolic arterial pressure (SAP) and respiration (RESP). It relies on construction of a multivariate embedding space, optimization of the embedding dimension and a procedure allowing the selection of the components most suitable to form the multivariate embedding space. Moreover, it allows the comparison between linear model-based (MB) and nonlinear model-free (MF) techniques and between MF approaches exploiting local predictability (LP) and conditional entropy (CE). The framework was applied to study age-related modifications of complexity and causality in healthy humans in supine resting (REST) and during standing (STAND). We found that: 1) MF approaches are more efficient than the MB method when nonlinear components are present, while the reverse situation holds in presence of high dimensional embedding spaces; 2) the CE method is the least powerful in detecting age-related trends; 3) the association of HP complexity on age suggests an impairment of cardiac regulation and response to STAND; 4) the relation of SAP complexity on age indicates a gradual increase of sympathetic activity and a reduced responsiveness of vasomotor control to STAND; 5) the association from SAP to HP on age during STAND reveals a progressive inefficiency of baroreflex; 6) the reduced connection from HP to SAP with age might be linked to the progressive exploitation of Frank-Starling mechanism at REST and to the progressive increase of peripheral resistances during STAND; 7) at REST the diminished association from RESP to HP with age suggests a vagal withdrawal and a gradual uncoupling between respiratory activity and heart; 8) the weakened connection from RESP to SAP with age might be related to the progressive increase of left ventricular thickness and vascular stiffness and to the gradual decrease of respiratory sinus arrhythmia.

We propose an integrated approach based on uniform quantization over a small number of levels for the evaluation and characterization of complexity of a process. This approach integrates information-domain analysis based on entropy rate, local nonlinear prediction, and pattern classification based on symbolic analysis. Normalized and non-normalized indexes quantifying complexity over short data sequences ( approximately 300 samples) are derived. This approach provides a rule for deciding the optimal length of the patterns that may be worth considering and some suggestions about possible strategies to group patterns into a smaller number of families. The approach is applied to 24 h Holter recordings of heart period variability derived from 12 normal (NO) subjects and 13 heart failure (HF) patients. We found that: (i) in NO subjects the normalized indexes suggest a larger complexity during the nighttime than during the daytime; (ii) this difference may be lost if non-normalized indexes are utilized; (iii) the circadian pattern in the normalized indexes is lost in HF patients; (iv) in HF patients the loss of the day-night variation in the normalized indexes is related to a tendency of complexity to increase during the daytime and to decrease during the nighttime; (v) the most likely length L of the most informative patterns ranges from 2 to 4; (vi) in NO subjects classification of patterns with L=3 indicates that stable patterns (i.e., those with no variations) are more present during the daytime, while highly variable patterns (i.e., those with two unlike variations) are more frequent during the nighttime; (vii) during the daytime in HF patients, the percentage of highly variable patterns increases with respect to NO subjects, while during the nighttime, the percentage of patterns with one or two like variations decreases.

Nollo, Giandomenico, Luca Faes, Alberto Porta, Renzo Antolini, and Flavia Ravelli. Exploring directionality in spontaneous heart period and systolic pressure variability interactions in humans: implications in the evaluation of baroreflex gain. Am J Physiol Heart Circ Physiol 288: H1777-H1785, 2005. First published December 16, 2004; doi:10.1152/ajpheart.00594.2004.-Although in physiological conditions RR interval and systolic arterial pressure (SAP) are likely to interact in a closed loop, the traditional cross-spectral analysis cannot distinguish feedback (FB) from feedforward (FF) influences. In this study, a causal approach was applied for calculating the coherence from SAP to RR (K s-r) and from RR to SAP (Kr-s) and the gain and phase of the baroreflex transfer function. The method was applied, compared with the noncausal one, to RR and SAP series taken from 15 healthy young subjects in the supine position and after passive head-up tilt. For the low frequency (0.04 -0.15 Hz) spectral component, the enhanced FF coupling (K r-s ϭ 0.59 Ϯ 0.21, significant in 14 subjects) and the blunted FB coupling (K s-r ϭ 0.17 Ϯ 0.17, significant in 4 subjects) found at rest indicated the prevalence of nonbaroreflex mechanisms. The tilt maneuver recovered FB influences (K s-r ϭ 0.47 Ϯ 0.16, significant in 14 subjects), which were stronger than FF interactions (K s-r ϭ 0.34 Ϯ 0.19, significant in 9 subjects). At the respiratory frequency, the RR-SAP regulation was balanced at rest (K s-r ϭ 0.30 Ϯ 0.18 and Kr-s ϭ 0.29 Ϯ 0.20, significant in 11 and 8 subjects) and shifted toward FB mechanisms after tilt (K s-r ϭ 0.35 Ϯ 0.19 and Kr-s ϭ 0.19 Ϯ 0.11, significant in 14 and 8 subjects). The causal baroreflex gain estimates were always lower than the corresponding noncausal values and decreased significantly from rest to tilt in both frequency bands. The tilt-induced increase of the phase lag from SAP to RR suggested a shift from vagal to sympathetic modulation. Thus the importance of nonbaroreflex interactions pointed out the necessity of accounting for causality in the cross-spectral analysis of the interactions between cardiovascular variables in healthy humans.cross-spectral analysis; coherence and transfer function; cardiovascular regulation; feedback and feedforward mechanisms; nonbaroreflex interactions SINCE THE INTRODUCTION of the notion that cardiovascular oscillations may be indicative of autonomic nervous system status (3), the analysis of the spontaneous fluctuations in heart rate and blood pressure has been extensively used as a probe for cardiovascular control mechanisms in humans. Particularly, the observation that oscillations in the heart period (RR interval) and the systolic arterial pressure (SAP) are correlated around 0.1 Hz and at the frequency of respiration has prompted many researchers to focus on the interrelationship between these two signals. In this context, the cross-spectral analysis of RR interval and SAP variability series constitutes one of the most widespread tools used to investigate on the coupl...