Model-free quantification of time-series predictability

Garland, Joshua; James, Ryan G.; Bradley, Elizabeth

doi:10.1103/physreve.90.052910

Cited by 78 publications

(96 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dynamical differences are visually apparent from the traces in Figure 3; they are mathematically apparent from nonlinear time-series analysis of embeddings of those data 40 , as well as in calculations of the information content of the two signals. Among other things, 403.gcc has much less predictive structure than col major and is thus much harder to forecast 16 . These attributes make this a useful pair of experiments for an exploration of the utility of reduced-order forecasting.…”

Section: B Experimental Data: Computer Performance Dynamicsmentioning

confidence: 99%

“…Even this simple strategy-which, as described in more detail in Section II B, builds predictions by looking for the nearest neighbor of a given point and taking that neighbor's observed path as the forecast-works quite well for forecasting nonlinear dynamical systems. LMA and similar methods have been used successfully to forecast measles and chickenpox outbreaks 11 , marine phytoplankton populations 11 , performance dynamics of a running computer [14][15][16] , the fluctuations in a far-infrared laser 2,10 , and many more.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction in projection

Garland

Bradley

2015

Chaos: An Interdisciplinary Journal of Nonlinear Science

Self Cite

View full text Add to dashboard Cite

Prediction models that capture and use the structure of state-space dynamics can be very effective. In practice, however, one rarely has access to full information about that structure, and accurate reconstruction of the dynamics from scalar time-series data-e.g., via delaycoordinate embedding-can be a real challenge. In this paper, we show that forecast models that employ incomplete embeddings of the dynamics can produce surprisingly accurate predictions of the state of a dynamical system. In particular, we demonstrate the effectiveness of a simple near-neighbor forecast technique that works with a two-dimensional embedding. Even though correctness of the topology is not guaranteed for incomplete reconstructions like this, the dynamical structure that they capture allows for accurate predictions-in many cases, even more accurate than predictions generated using a full embedding. This could be very useful in the context of real-time forecasting, where the human effort required to produce a correct delay-coordinate embedding is prohibitive. LEAD PARAGRAPHPrediction models constructed from state-space dynamics have a long and rich history, dating back to roulette and beyond. A major stumbling block in the application of these models in real-world situations is the need to reconstruct the dynamics from scalar time-series data-e.g., via delay-coordinate embedding. This procedure, which is the topic of a large and active body of literature, involves estimation of two free parameters: the dimension m of the reconstruction space and the delay, τ , between the observations that make up the coordinates in that space. Estimating good values for these parameters is not trivial; it requires the proper mathematics, attention to the data requirements, computational effort, and expert interpretation of the results of the calculations. This is a major challenge if one is interested in real-time forecasting, especially when the systems involved operate a) Electronic mail: joshua.garland@colorado.edu b) Electronic mail: lizb@colorado.edu on fast time scales. In this paper, we show that the full effort of delay-coordinate embedding is not always necessary when one is building forecast models, and can indeed be overkill. Using synthetic time-series data generated from the Lorenz-96 atmospheric model and real data from a computer performance experiment, we demonstrate that a two-dimensional embedding of scalar timeseries data from a dynamical system gives simple forecast methods enough traction to generate accurate predictions of the future course of those dynamics-sometimes even more accurate than predictions created using the full embedding. Since incomplete embeddings do not preserve the topology of the full dynamics, this is interesting from a mathematical standpoint. It is also potentially useful in practice. This reduced-order forecasting strategy involves only one free parameter (τ ), good values for which, we believe, can be estimated 'on the fly' using information-theoretic and/or machine-learning algorithms. As suc...

show abstract

Section: B Experimental Data: Computer Performance Dynamicsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Prediction in projection

Garland

Bradley

2015

Chaos: An Interdisciplinary Journal of Nonlinear Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…, Garland et al. ). In principle, intrinsic predictability has the potential to indicate whether the model, data, or system are limiting realized predictability.…”

Section: Introductionmentioning

confidence: 99%

The intrinsic predictability of ecological time series and its potential to guide forecasting

Pennekamp

Iles

Garland

et al. 2019

Ecological Monographs

Self Cite

103

View full text Add to dashboard Cite

Successfully predicting the future states of systems that are complex, stochastic, and potentially chaotic is a major challenge. Model forecasting error (FE) is the usual measure of success; however model predictions provide no insights into the potential for improvement. In short, the realized predictability of a specific model is uninformative about whether the system is inherently predictable or whether the chosen model is a poor match for the system and our observations thereof. Ideally, model proficiency would be judged with respect to the systems’ intrinsic predictability, the highest achievable predictability given the degree to which system dynamics are the result of deterministic vs. stochastic processes. Intrinsic predictability may be quantified with permutation entropy (PE), a model‐free, information‐theoretic measure of the complexity of a time series. By means of simulations, we show that a correlation exists between estimated PE and FE and show how stochasticity, process error, and chaotic dynamics affect the relationship. This relationship is verified for a data set of 461 empirical ecological time series. We show how deviations from the expected PE–FE relationship are related to covariates of data quality and the nonlinearity of ecological dynamics. These results demonstrate a theoretically grounded basis for a model‐free evaluation of a system's intrinsic predictability. Identifying the gap between the intrinsic and realized predictability of time series will enable researchers to understand whether forecasting proficiency is limited by the quality and quantity of their data or the ability of the chosen forecasting model to explain the data. Intrinsic predictability also provides a model‐free baseline of forecasting proficiency against which modeling efforts can be evaluated.

show abstract

“…Due to its focus on probability distributions, it allows one to compare dissimilar systems (e.g., species abundance to ground state configurations of a spin system) and has found many successes in the physical, biological and social sciences [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] far outside its original domain of communication. Often, the issue on which it is brought to bear is discovering and quantifying dependencies [20][21][22][23][24][25].…”

Section: Introductionmentioning

confidence: 99%

Multivariate Dependence beyond Shannon Information

James

Crutchfield

2017

Entropy

Self Cite

View full text Add to dashboard Cite

Accurately determining dependency structure is critical to understanding a complex system's organization. We recently showed that the transfer entropy fails in a key aspect of this-measuring information flow-due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that Shannon information measures (entropy and mutual information, in their conditional and multivariate forms) can fail to accurately ascertain multivariate dependencies due to their conflation of qualitatively different relations among variables. This has broad implications, particularly when employing information to express the organization and mechanisms embedded in complex systems, including the burgeoning efforts to combine complex network theory with information theory. Here, we do not suggest that any aspect of information theory is wrong. Rather, the vast majority of its informational measures are simply inadequate for determining the meaningful relationships among variables within joint probability distributions. We close by demonstrating that such distributions exist across an arbitrary set of variables.

show abstract

Model-free quantification of time-series predictability

Cited by 78 publications

References 44 publications

Prediction in projection

Prediction in projection

The intrinsic predictability of ecological time series and its potential to guide forecasting

Multivariate Dependence beyond Shannon Information

Contact Info

Product

Resources

About