A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

Platt, Jason A.; Penny, Stephen G.; Smith, Timothy A.; Chen, Tse‐Chun

doi:10.1016/j.neunet.2022.06.025

Cited by 15 publications

(13 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…single layer autoregressive and recurrent neural network architectures, mainly because it has been shown that they can successfully emulate low dimensional chaotic dynamics over multiple Lyapunov timescales (Gauthier et al, 2021;Pathak et al, 2017;Platt et al, 2022;Vlachas et al, 2020). We implemented a multi-dimensional parallelization scheme based on the concept introduced by Pathak et al (2018) and similar to that of Arcomano et al (2020) in order to scale up these architectures and test them in high dimensional systems.…”

Section: Discussionmentioning

confidence: 99%

“…In this study, our primary interest was to shed light on the spatial scales that can be resolved by single layer autoregressive and recurrent neural network emulators in order to better understand the effective resolution that could be achieved in weather and climate applications. We used two relatively simple, single layer autoregressive and recurrent neural network architectures, mainly because it has been shown that they can successfully emulate low dimensional chaotic dynamics over multiple Lyapunov timescales (Gauthier et al., 2021; Pathak et al., 2017; Platt et al., 2022; Vlachas et al., 2020). We implemented a multi‐dimensional parallelization scheme based on the concept introduced by Pathak et al.…”

Section: Discussionmentioning

confidence: 99%

“…A key aspect in machine learning is normalizing input data before passing it to the model. Experiments from Platt et al (2022) showed, however, that the standard approach to normalizing data can be detrimental to prediction skill. By "standard approach," we mean Figure A2 compares the prediction skill when these two normalization strategies are used.…”

Section: A3 Data Normalizationmentioning

confidence: 99%

“…although we do not form the inverse explicitly. We instead use the solve function from SciPy's linear algebra module (Virtanen et al, 2020), based on testing shown in Appendix C of Platt et al (2022).…”

Section: Linear Readout and Trainingmentioning

confidence: 99%

“…that is, 𝐴𝐴 𝐴 𝐴𝐴𝑖𝑖 and SD i are the mean and standard deviation taken from the training data separately over each channel of data, indexed by i. The key takeaway from Platt et al (2022) is that by using separate normalization values for each channel, the covarying relationships between the data are destroyed and the reservoir cannot learn the true dynamics. The authors propose to normalize with the average and range of the data, computed over the length of the training data and over all channels…”

Section: Appendix A: Matrix and Data Normalization For Echo State Net...mentioning

confidence: 99%

See 4 more Smart Citations

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

Smith,

Penny,

Platt

et al. 2023

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.

show abstract