Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations

Abstract

Entropy measures are widely applied to quantify the complexity of dynamical systems in diverse fields. However, the practical application of entropy methods is challenging, due to the variety of entropy measures and estimators and the complexity of real-world time series, including nonstationarities and long-range correlations (LRC). We conduct a systematic study on the performance, bias, and limitations of three basic measures (entropy, conditional entropy, information storage) and three traditionally used estimators (linear, kernel, nearest neighbor). We investigate the dependence of entropy measures on estimator- and process-specific parameters, and we show the effects of three types of nonstationarities due to artifacts (trends, spikes, local variance change) in simulations of stochastic autoregressive processes. We also analyze the impact of LRC on the theoretical and estimated values of entropy measures. Finally, we apply entropy methods on heart rate variability data from subjects in different physiological states and clinical conditions. We find that entropy measures can only differentiate changes of specific types in cardiac dynamics and that appropriate preprocessing is vital for correct estimation and interpretation. Demonstrating the limitations of entropy methods and shedding light on how to mitigate bias and provide correct interpretations of results, this work can serve as a comprehensive reference for the application of entropy methods and the evaluation of existing studies.

Efficient Computation of Multiscale Entropy over Short Biomedical Time Series Based on Linear State-Space Models

“…These results suggest that the simultaneous presence of multiple oscillatory mechanisms tends to produce more complex dynamics than in the case of single mechanisms, with a complexity degree that increases with the strength of the stochastic oscillations and with the mismatch of their frequency. These findings are supported by the results of recent studies [32,33] and are observed over a range of time scales that comprises the characteristic periods of all oscillations. Figure 2 reports the results of the practical estimation of complexity over short realizations of the simulations, performed using the refined approach and the linear approach to MSE computation.…”

“…The behavior of MSE for oscillations with fixed amplitude ( 1 = 0.8) and varying frequency is more complicated (Figure 1(b)). At scale one ( = 0.5), the MSE is the same if the frequency of the oscillatory dynamics, 1 , has the same distance from half the Nyquist frequency of 0.25 Hz and decreases with such a distance; the same symmetric behavior, with maximum complexity at half the Nyquist frequency, was observed in [33]. Then, the multiscale behavior of the complexity measure is related to the frequency of the stochastic oscillation in a way such that faster oscillations are removed more easily than slower oscillations, and thus MSE reaches its higher values for higher values of (see the trends of 1 = 0.4 versus 1 = 0.1 and 1 = 0.3 versus 1 = 0.2, in Figure 1(b)).…”

“…On the contrary, RMSE estimates are strongly biased at all time scales and also exhibit a substantial variance that increases dramatically with the time scale (Figures 2(b) and 2(d)); note that RMSE could not be computed at the higher time scales ( = 0.05, = 0.025) due to the limited number of data points. The poor performance of RMSE confirms the known difficulty of yielding accurate model-free complexity estimates using the SampEn estimator applied to short time series [33,34].…”

Assessment of ECG and respiration recordings from simulated emergency landings of ultra light aircraft

“…Entropy measures 34 were applied to evaluate additional information from signal. To evaluate entropy of HRV and respiration approximate entropy, sample entropy, and conditional entropy was used.…”

Networked information interactions of epileptic EEG based on symbolic transfer entropy

“…The non-parametric method does not require priori assumptions of whether linear or nonlinear dependence although it is mathematically equivalent to Granger Causality [23,24]. However, the estimation of transition probabilities from the data is not trivial, and solutions for this issue include entropy estimators [25], like the step kernel estimation which is employed by T. Schreiber [16] in the original paper. Another alternative solution is symbolic time series analysis [26] that transforms raw time series with continuous distributions into symbolic sequence containing discretized symbols from some alphabet [17,27] to simplify the calculation of probability distributions.…”