Approximate entropy as a measure of system complexity.

Abstract

Techniques to determine changing system complexity from data are evaluated. Convergence of a frequently used correlation dimension algorithm to a finite value does not necessarily imply an underlying deterministic model or chaos. Analysis of a recently developed family of formulas and statistics, approximate entropy (ApEn), suggests that ApEn can classify complex systems, given at least 1000 data values in diverse settings that include both deterministic chaotic and stochastic processes. The capability to discern changing complexity from such a relatively small amount of data holds promise for applications of ApEn in a variety of contexts.In an effort to understand complex phenomena, investigators throughout science are considering chaos as a possible underlying model. Formulas have been developed to characterize chaotic behavior, in particular to encapsulate properties of strange attractors that represent long-term system dynamics. Recently it has become apparent that in many settings nonmathematicians are applying new "formulas" and algorithms to experimental time-series data prior to careful statistical examination. One sees numerous papers concluding the existence of deterministic chaos from data analysis (e.g., ref. 1) and including "error estimates" on dimension and entropy calculations (e.g., ref. 2). While mathematical analysis of known deterministic systems is an interesting and deep problem, blind application of algorithms is dangerous, particularly so here. Even for low-dimensional chaotic systems, a huge number of points are needed to achieve convergence in these dimension and entropy algorithms, though they are often applied with an insufficient number of points. Also, most entropy and dimension definitions are discontinuous to system noise. Furthermore, one sees interpretations of dimension calculation values that seem to have no general basis in fact-e.g., number of free variables and/or differential equations needed to model a system.The purpose of this paper is to give a preliminary mathematical development of a family of formulas and statistics, approximate entropy (ApEn), to quantify the concept of changing complexity. We ask three basic questions: (i) Can one certify chaos from a converged dimension (or entropy) calculation? (ii) If not, what are we trying to quantify, and what tools are available? (iii) If we are trying to establish that a measure of system complexity is changing, can we do so with far fewer data points needed, and more robustly than with currently available tools?I demonstrate that one can have a stochastic process with correlation dimension 0, so the answer to i is No. It appears that stochastic processes for which successive terms are correlated can produce finite dimension values. A "phase space plot" of consecutive terms in such instances would then demonstrate correlation and structure. This implies neither a deterministic model nor chaos. Compare this to figures 4 a and b of Babloyantz and Destexhe (1).If one cannot hope to establish chaos, presumably one is...

Analysis of regularity in the EEG background activity of Alzheimer's disease patients with Approximate Entropy

“…Normalizing r in this manner gives ApEn a translation and scale invariance, in that it remains unchanged under uniform process magnification, reduction, or constant shift to higher or lower values (Pincus, 2001). Moreover, several studies (Pincus, 1991;Pincus and Keefe, 1992) have demonstrated that these input parameters produce good statistical reproducibility for ApEn for time series of length N≥60, as considered herein. For this pilot study, ApEn was estimated with m=1 and r=0.2 times the SD of the original data sequence.…”

“…ApEn was introduced as a quantification of regularity in sequences and time series data, initially motivated by applications to relatively short, noisy data sets (Pincus, 1991). It is scale invariant and model independent, evaluates both dominant and subordinated patterns in data, and discriminates series for which clear feature recognition is difficult.…”

“…The number of matches can be increased by choosing small m (short templates) and large r (wide tolerance). However, there are penalties for criteria that are too relaxed (Pincus, 1991). Pincus (2001) has suggested to estimate ApEn with parameter values of m=1, m=2 and r=0.1, 0.15, 0.2 and 0.25 times the SD of the original data sequence {x(n)}.…”

“…Unfortunately, K-S entropy was not developed for statistical applications and diverges to a value of infinity when the signal is contaminated by the slightest noise. A practical solution to this problem has been put forward using a recently developed family of statistics named Approximate Entropy (ApEn) (Pincus, 1991). Although there are many other entropy estimators, several properties of ApEn facilitate its utility for empirical time series analysis of the sort of EEGs: ApEn is nearly unaffected by noise below a de facto specified filter level (r), it can be applied to time series of 50 or more points with good reproducibility; it is finite for stochastic, noisy deterministic and composite processes, and increasing values of ApEn correspond to more irregularity or to increasing complexity in the time series (Pincus 2001).…”

Loss of beta-cell mass leads to a reduction of pulse mass with normal periodicity, regularity and entrainment of pulsatile insulin secretion in G�ttingen minipigs

“…The regularity of serum insulin concentration time series was assessed by application of Approximate Entropy (ApEn), which is a model independent statistic [27]. ApEn has been introduced to discriminate between insulin secretory profiles in glucose tolerant first degree relatives of Type 2 diabetic patients compared with healthy people [8].…”