Discriminating chaotic and stochastic time series using permutation entropy and artificial neural networks

Abstract

Extracting relevant properties of empirical signals generated by nonlinear, stochastic, and high-dimensional systems is a challenge of complex systems research. Open questions are how to differentiate chaotic signals from stochastic ones, and how to quantify nonlinear and/or high-order temporal correlations. Here we propose a new technique to reliably address both problems. Our approach follows two steps: first, we train an artificial neural network (ANN) with flicker (colored) noise to predict the value of the parameter, $$\alpha$$ α , that determines the strength of the correlation of the noise. To predict $$\alpha$$ α the ANN input features are a set of probabilities that are extracted from the time series by using symbolic ordinal analysis. Then, we input to the trained ANN the probabilities extracted from the time series of interest, and analyze the ANN output. We find that the $$\alpha$$ α value returned by the ANN is informative of the temporal correlations present in the time series. To distinguish between stochastic and chaotic signals, we exploit the fact that the difference between the permutation entropy (PE) of a given time series and the PE of flicker noise with the same $$\alpha$$ α parameter is small when the time series is stochastic, but it is large when the time series is chaotic. We validate our technique by analysing synthetic and empirical time series whose nature is well established. We also demonstrate the robustness of our approach with respect to the length of the time series and to the level of noise. We expect that our algorithm, which is freely available, will be very useful to the community.

Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

“…In our methodology, we not only apply dimensionality reduction but also use a fully stochastic "reference" FN time series: we compare the value ofS of the time series of interest with that of a FN time series generated with α = α e . We have shown that the entropy difference, Ω, may provide good contrast for distinguishing fully stochastic time series from a time series with a degree of determinism [9].…”

“…We have recently proposed a new method for estimating the strength of the temporal correlations in a given time series, which uses flicker noise (FN), a fully stochastic process, as the reference model [9]. A FN time series, x FN α (t), is characterized by a power spectrum P( f ) ∝ 1/ f α , with α being a parameter that quantifies the temporal correlations present in the signal [10].…”

“…A FN time series, x FN α (t), is characterized by a power spectrum P( f ) ∝ 1/ f α , with α being a parameter that quantifies the temporal correlations present in the signal [10]. The method proposed in [9] combines the use of symbolic ordinal analysis [11,12] and machine learning (ML): We utilize the ordinal probabilities computed from FN time series generated with different α values as input features to a ML algorithm. The algorithm is trained to return the value of α e , which estimates the real α value of a FN time series from the D!…”

“…We have shown that the algorithm returns meaningful information even from time series that are very short: for the synthetic examples considered in [9], we could distinguish whether the dynamics is mainly chaotic or stochastic with only 100 data points. However, an open question is as follows: What are the limitations in terms of the level of chaos, the level of noise and the length of the time series?…”

“…In the implementation proposed in [9], the probabilities of the 720 ordinal patterns of length D = 6 (described in the next section) were used as input features to the ML algorithm. Then, these features were reduced to only two: the scalar value returned by the ML algorithm, α e , which quantifies the temporal correlations presented in the time series of interest x(t); and the permutation entropy,S (Equation ( 4)).…”