Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics

Vlachas, Pantelis R.; Pathak, Jaideep; Hunt, Brian R.; Sapsis, Themistoklis P.; Girvan, Michelle; Ott, Edward; Koumoutsakos, Petros

doi:10.1016/j.neunet.2020.02.016

Cited by 323 publications

(218 citation statements)

References 49 publications

Supporting

Mentioning

189

Contrasting

Order By: Relevance

“…with accuracy comparable to LETKF-Ext, if we have perfect observations. This result is consistent with Chattopadhyay et al 2019, Pathak et al (2017) or P. R. Vlachas et al (2020), and we can expect that RC has a potential to predict various kinds of spatio-temporal chaotic systems.…”

Section: Discussionsupporting

confidence: 93%

“…However, the interpolated data inevitably includes errors even if the observation data itself has no error, so it should be verified that RC can predict accurately by training data with some errors. Previous works such as Chattopadhyay et al, 2019, or P. R. Vlachas et al, 2020 have not considered the impact of error https://doi.org/10.5194/gmd-2020-211 Preprint. Discussion started: 21 August 2020 c Author(s) 2020.…”

Section: Discussionmentioning

confidence: 99%

“…However, Vlachas et al (2020) revealed that the prediction accuracy of RC is degraded when all of the state variables cannot be observed. It can be a serious problem since the observation sparsity is often the case in geosciences and the NWP systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Combining Ensemble Kalman Filter and Reservoir Computing to predict spatio-temporal chaotic systems from imperfect observations and models

Tomizawa¹,

Sawada²

2020

Preprint

View full text Add to dashboard Cite

Abstract. Prediction of spatio-temporal chaotic systems is important in various fields, such as Numerical Weather Prediction (NWP). While data assimilation methods have been applied in NWP, machine learning techniques, such as Reservoir Computing (RC), are recently recognized as promising tools to predict spatio-temporal chaotic systems. However, the sensitivity of the skill of the machine learning based prediction to the imperfectness of observations is unclear. In this study, we evaluate the skill of RC with noisy and sparsely distributed observations. We intensively compare the performances of RC and Local Ensemble Transform Kalman Filter (LETKF) by applying them to the prediction of the Lorenz 96 system. Although RC can successfully predict the Lorenz 96 system if the system is perfectly observed, we find that RC is vulnerable to observation sparsity compared with LETKF. To overcome this limitation of RC, we propose to combine LETKF and RC. In our proposed method, the system is predicted by RC that learned the analysis time series estimated by LETKF. Our proposed method can successfully predict the Lorenz 96 system using noisy and sparsely distributed observations. Most importantly, our method can predict better than LETKF when the process-based model is imperfect.

show abstract

Section: Discussionsupporting

confidence: 93%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Combining Ensemble Kalman Filter and Reservoir Computing to predict spatio-temporal chaotic systems from imperfect observations and models

Tomizawa¹,

Sawada²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…176 Another approach would be to use CNNs, for which backpropagation through time (BPTT) algorithms are used instead of RC computing. 176,177 Regularization procedures utilizing BPTT were also shown to be more effective in chaotic attractor reconstruction. 176,177 CNN-based methods may be better suited for the classification of time series gene expression dynamics, while RNN algorithms, such as RC, are best for time series forecasting/ attractor reconstruction.…”

Section: <mentioning

confidence: 99%

“…176,177 Regularization procedures utilizing BPTT were also shown to be more effective in chaotic attractor reconstruction. 176,177 CNN-based methods may be better suited for the classification of time series gene expression dynamics, while RNN algorithms, such as RC, are best for time series forecasting/ attractor reconstruction. 166 RC is computationally cheaper to train in the case of full-state information acquisition whereas, gated architectures are better for reduced order states/observables.…”

Section: <mentioning

confidence: 99%

A review of dynamical systems approaches for the detection of chaotic attractors in cancer networks

Uthamacumaran

2021

Patterns

View full text Add to dashboard Cite

Cancers are complex dynamical systems. They remain the leading cause of disease-related pediatric mortality in North America. To overcome this burden, we must decipher the state-space attractor dynamics of gene expression patterns and protein oscillations orchestrated by cancer stemness networks. The review provides an overview of dynamical systems theory to steer cancer research in pattern science. While most of our current tools in network medicine rely on statistical correlation methods, causality inference remains primitively developed. As such, a survey of attractor reconstruction methods and machine algorithms for the detection of causal structures applicable in experimentally derived time series cancer datasets is presented. A toolbox of complex systems approaches are discussed for reconstructing the signaling state space of cancer networks, interpreting causal relationships in their time series gene expression patterns, and assisting clinical decision making in computational oncology. As a proof of concept, the applicability of some algorithms are demonstrated on pediatric brain cancer datasets and the requirement of their time series analysis is highlighted. ll

show abstract

Physical Implementation of Reservoir Computing through Electrochemical Reaction

et al. 2021

View full text Add to dashboard Cite

Nonlinear dynamical systems serving reservoir computing enrich the physical implementation of computing systems. A method for building physical reservoirs from electrochemical reactions is provided, and the potential of chemical dynamics as computing resources is shown. The essence of signal processing in such systems includes various degrees of ionic currents which pass through the solution as well as the electrochemical current detected based on a multiway data acquisition system to achieve switchable and parallel testing. The results show that they have respective advantages in periodic signals and temporal dynamic signals. Polyoxometalate molecule in the solution increases the diversity of the response current and thus improves their abilities to predict periodic signals. Conversely, distilled water exhibits great computing power in solving a second‐order nonlinear problem. It is expected that these results will lead to further exploration of ionic conductance as a nonlinear dynamical system and provide more support for novel devices as computing resources.

show abstract

Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics

Cited by 323 publications

References 49 publications

Combining Ensemble Kalman Filter and Reservoir Computing to predict spatio-temporal chaotic systems from imperfect observations and models

Combining Ensemble Kalman Filter and Reservoir Computing to predict spatio-temporal chaotic systems from imperfect observations and models

A review of dynamical systems approaches for the detection of chaotic attractors in cancer networks

Physical Implementation of Reservoir Computing through Electrochemical Reaction

Contact Info

Product

Resources

About