The intrinsic predictability of ecological time series and its potential to guide forecasting

Abstract

Successfully predicting the future states of systems that are complex, stochastic, and potentially chaotic is a major challenge. Model forecasting error (FE) is the usual measure of success; however model predictions provide no insights into the potential for improvement. In short, the realized predictability of a specific model is uninformative about whether the system is inherently predictable or whether the chosen model is a poor match for the system and our observations thereof. Ideally, model proficiency would be judged with respect to the systems’ intrinsic predictability, the highest achievable predictability given the degree to which system dynamics are the result of deterministic vs. stochastic processes. Intrinsic predictability may be quantified with permutation entropy (PE), a model‐free, information‐theoretic measure of the complexity of a time series. By means of simulations, we show that a correlation exists between estimated PE and FE and show how stochasticity, process error, and chaotic dynamics affect the relationship. This relationship is verified for a data set of 461 empirical ecological time series. We show how deviations from the expected PE–FE relationship are related to covariates of data quality and the nonlinearity of ecological dynamics. These results demonstrate a theoretically grounded basis for a model‐free evaluation of a system's intrinsic predictability. Identifying the gap between the intrinsic and realized predictability of time series will enable researchers to understand whether forecasting proficiency is limited by the quality and quantity of their data or the ability of the chosen forecasting model to explain the data. Intrinsic predictability also provides a model‐free baseline of forecasting proficiency against which modeling efforts can be evaluated.

Rapid and positive responses of plants to lower precipitation predictability

“…During 2013-2015, precipitation falling at the field site exhibited no seasonality (electronic supplementary material, figure S3a), the average annual precipitation at the field site was 916 mm (2012-2015), and maximum annual precipitation registered at the field site in the last 10 years was 1265 mm m −2 . The intrinsic predictability of the total precipitation (natural + supplemental) estimated by permutation entropy [2] was 0.77 in the M-plots, which was equal to the predictability of the natural precipitation of the study site. In L-plots, it was 0.86, and thus, 11.2% lower than that of the M-plots [19].…”

On the predictability of infectious disease outbreaks

“…In order to study the predictability of diseases in a comparative framework, which also permits stochasticity and model nonstationarity, we employ permutation entropy as a model-free measure of time-series predictability [31][32][33] . This measure, i.e permutation entropy, is ideal because-in addition to being a model independent metric of predictability-recent work has demonstrated that it correlates strongly with known limits to forecasting in dynamical systems, e.g., models where we can measure Lyapunov stability [31][32][33] and can be transformed into an estimate of Kolmogorov-Sinai entropy 34 . Additionally, recent studies by Pennekamp et al 33 and Garland et al 35 demonstrated that permutation entropy correlated strongly with forecast accuracy for ecological models and with anomalies in climatological data.…”

Anomaly Detection in Paleoclimate Records Using Permutation Entropy

“…Since its conception, PE has been shown to converge to the Kolmogorov–Sinai entropy [ 15 , 16 ] or the metric entropy [ 29 , 30 ] under a variety of conditions and depending on the features (e.g., stationary, ergodic) of the underlying generating process. It has also been shown to correlate with intrinsic predictability in a variety of time series [ 13 , 14 ]. For the purposes of this paper, we simply view permutation entropy as a measure of the temporal complexity of a time series and we treat abrupt changes in that complexity as a signal of possible anomalies, naturally occurring or data related.…”

“…In the context of a time series, this translates to a measure of how information propagates forward temporally. This has implications for predictability [ 13 , 14 ], among other things; indeed, these measures have been shown to converge to the Kolmogorov–Sinai entropy under suitable conditions [ 15 , 16 ], as described in Section 2.2 . The goal here is not to measure the complexity of paleoclimate data in any formal way, however.…”