Small-shuffle surrogate data: Testing for dynamics in fluctuating data with trends

Abstract

We describe a method for identifying dynamics in irregular time series ͑short term variability͒. The method we propose focuses attention on the flow of information in the data. We can apply the method even for irregular fluctuations which exhibit long term trends ͑periodicities͒: situations in which previously proposed surrogate methods would give erroneous results. The null hypothesis addressed by our algorithm is that irregular fluctuations are independently distributed random variables ͑in other words, there is no short term dynamics͒. The method is demonstrated for numerical data generated by known systems, and applied to several actual time series. There are many natural phenomena that show irregular fluctuations ͑short term variability͒. The question of whether the fluctuations are random or not is an old one and extremely important. If the fluctuations are not random, then they are due to some kind of dynamical structure and then it might be possible to build deterministic models or model systems from the time series. Clearly, such models are of immense value for both understanding and predicting the time series. To investigate whether the data can be fully described by independent and identically distributed ͑IID͒ random variables, the random-shuffle surrogate ͑RSS͒ method has been proposed ͓1͔. Although this method is effective for time series with no trends ͑periodicities͒ like that shown in Figs. 1͑a͒ and 1͑b͒, the algorithm is ineffective for data exhibing slow trends or periodicities ͓see Figs. 1͑c͒ and 1͑d͔͒. Such cases are theoretically incompatible with the assumption of the RSS method as well as other linear surrogate tests ͓1,2͔. There is currently no method which can tackle this problem. In this Communication, to investigate whether there is dynamics in data which also exhibits irregular fluctuations, we introduce such a method.The basic premise of this technique is that if irregular fluctuations are not random, then there is some kind of underlying dynamical system: whatever trending is contaminating the data. In such a case, the data index ͑order͒ itself has important implications irrespective of whether time series are linear or nonlinear. Hence, whenever the index changes, the flow of information also changes and the resultant time series no longer reflects the original dynamics. We focus our attention on this point and propose a surrogate method using this idea. The purpose of our method is to distinguish between irregular fluctuations with or without dynamics.After describing our technique, we will present our choice of discriminating statistic. Then, we will apply this algorithm to two cases using simulated time series data. One case is that data have no trend ͑this case can also be adequately addressed with the standard surrogate methods͒. The other case is that data have trends ͑this case is not consistent with existing surrogate techniques͒. In each case, the data we use are both noise free and subsequently contaminated by 10% Gaussian observational noise. Also, we apply the method t...

Testing for Linear and Nonlinear Gaussian Processes in Nonstationary Time Series

“…Theiler's FT and AAFT methods cannot create surrogate data sufficiently similar to time series characterized by nonstationary behavior. In such situation, the surrogates produced by these methods are affected by the amplitude variation in the Fourier transform, resulting in surrogates completely different from the original time series Nakamura & Small, 2005].…”

“…According to the authors [Nakamura & Small, 2005], this method can be used to investigate irregular fluctuations in time series, once it destroys local structures or correlations and keeps the global behavior, such as trend. Hence, the null hypothesis addressed by this new method is that the time series consists of a underlying (slow) trend and that the fast dynamics are random.…”

“…Autocorrelation Function (ACF) [Provanzale et al, 1992] and the Average Mutual Information (AMI) [Abarbanel, 1996;Nakamura & Small, 2005;; (iii) hypothesis test considering the discriminating statistics obtained using AMI.…”

“…The main reason behind this is that by applying the Fourier transform on nonstationary time series, the difference between the first and last observations, caused by the trend, produces spurious high frequencies, as a consequence, inconsistent surrogates are produced. Other authors, such as Nakamura & Small, 2005], have also addressed this issue, but they consider parametric approaches to produce surrogate data which strongly depends on data being analyzed. On the other hand, this paper relies on the Empirical Mode Decomposition (EMD) method to automatically produce surrogate data.…”

Recurrence network measures for hypothesis testing using surrogate data: Application to black hole light curves

“…However, the surrogate data generated by this method are influenced by a cut-off frequency. In addition, there are also some other types of surrogate data testing reported in the literature, such as, cycle shuffle surrogates [12], surrogates for testing pseudoperiodic time series [13] and even recurrence based surrogates [14], with each scheme found useful in particular contexts. In this work, we apply the IAAFT scheme to generate surrogate data using the TISEAN package [15].…”