Limits to ecological forecasting: Estimating uncertainty for critical transitions with deep learning

Abstract

In the current age of a rapidly changing environment, it is becoming increasingly important to study critical transitions and how to best anticipate them. Critical transitions pose extremely challenging forecasting problems, which necessitate informative uncertainty estimation rather than point forecasts. In this study, we apply some of the most cutting edge deep learning methods for probabilistic time series forecasting to several classic ecological models that examine critical transitions. Our analysis focuses on three different simulated examples of critical transitions: a Hopf bifurcation, a saddle‐node bifurcation and a stochastic transition. For each scenario, we compare the forecasts from four deep learning models, long‐short term memory networks, gated recurrent unit networks, lock recurrent neural networks and transformers, to forecasts from an ARIMA model and a MCMC estimated model that is given the true transition dynamics. We found that the deep learning models were able to perform comparably to the idealized MCMC model on the stochastic transition case, and generally in between the MCMC and ARIMA models on the Hopf and saddle‐node bifurcation examples. Our results establish that deep learning methods warrant further exploration on the challenging class of critical transition forecasting problems.

A Multi‐Model Ensemble of Baseline and Process‐Based Models Improves the Predictive Skill of Near‐Term Lake Forecasts

“…For individual 1–14 days‐ahead forecasts at specific horizons and depths, individual models outperformed the MMEs (Figure 7), accounting for >96% of the best forecasts at 1 m and >91% at 8 m (Figure S4 in Supporting Information ). Each model captures slightly different dynamics of the mechanistic processes controlling reservoir water temperature and therefore performed optimally under different conditions (Lapeyrolerie & Boettiger, 2023). This was also observed in a multi‐model river forecasting study in which individual models alternately performed best in predicting different stages, phases, or mechanisms of rainfall‐runoff (Abrahart & See, 2002) and a penguin population forecasting study in which a range of models differentially captured inter‐annual and inter‐species variability (Humphries et al., 2018).…”

Using neural ordinary differential equations to predict complex ecological dynamics from population density data

“…To create these ARIMA models, we use the auto.arima function of the forecast package in R v. 4.2 [49]. We also compare our models with state-of-the-art, non-parametric algorithms in ecological forecasting, including EDM [50] and long-short term memory (LSTM) [26]. We create EDM models in R using the rEDM package and LSTM models using TensorFlow using the default parameters for both algorithms.…”

Synthesizing forecasts to inform decision‐making and advance ecological theory

“…Examples include modelling non‐linearities in response variables, modelling unobserved variables, and quantifying uncertainty. Many of the papers in this special feature deal with these ubiquitous challenges of forecasting, but address them with a range of modelling approaches from machine learning (Clark & Wells, 2023; Lapeyrolerie & Boettiger, 2022) to process‐based models (Cameron et al, 2022). To illustrate this, we consider how some of the papers in this special feature approach estimates of unobserved variables or uncertainty with myriad approaches.…”

“…Taking an entirely different modelling approach to Cameron et al (2022), Lapeyrolerie and Boettiger (2022) consider uncertainty estimation with deep learning methods in the context of forecasting abrupt changes in ecosystem dynamics (i.e. critical transitions).…”

“…A major shortcoming of many deep learning models for forecasting, however, is that it is difficult to estimate model uncertainty from them. Lapeyrolerie and Boettiger (2022) explore the performance of several deep learning algorithms that can account for model uncertainty to see how well they are able to forecast different simulated examples of critical transitions in comparison to forecasts produced by Markov chain Monte Carlo (MCMC) and autoregressive integrated moving average (ARIMA) models given the ‘true’ transition dynamics. The deep learning methods capable of estimating uncertainty perform comparably to the MCMC and ARIMA approaches, further distinguishing deep learning as a powerful tool for forecasting critical transitions.…”