Predicting dynamical system evolution with residual neural networks

Chashchin, Artem; Botchev, Mikhail; Oseledets, Ivan; Ovchinnikov, George

doi:10.48550/arxiv.1910.05233

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, deep neural networks have been replacing standard regression methods for dynamical systems modeling. Besides specific approaches like Deep Koopman model [18,19] and Physics-Informed neural networks [20,21,22,23], various universal architectures including ResNet [24], ODEnet [25,26,27,28] and recurrent models [29,30,31,32,33,34,35] have been applied to the dynamics modeling of chaotic systems. Still, none of the mentioned methods can preserve the invariant set of the dynamical system without additional modifications.…”

Section: Related Workmentioning

confidence: 99%

Tensorized Transformer for Dynamical Systems Modeling

Shalova,

Oseledets

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The identification of nonlinear dynamics from observations is essential for the alignment of the theoretical ideas and experimental data. The last, in turn, is often corrupted by the side effects and noise of different natures, so probabilistic approaches could give a more general picture of the process. At the same time, high-dimensional probabilities modeling is a challenging and data-intensive task. In this paper, we establish a parallel between the dynamical systems modeling and language modeling tasks. We propose a transformer-based model that incorporates geometrical properties of the data and provide an iterative training algorithm allowing the fine-grid approximation of the conditional probabilities of high-dimensional dynamical systems.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Tensorized Transformer for Dynamical Systems Modeling

Shalova,

Oseledets

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…They utilized a physics-based regularization as a penalty term to the optimization cost function to enforce physics into the training. Apart from LSTM, other machine learning algorithms such as reservoir computing have been used for modeling chaotic dynamical systems [10,67] and residual network for predicting dynamical system evolution [68,69]. In a recent study, Vlachas et al [70] investigated the performance of LSTM trained with backpropagation through time and reservoir computing for long term forecasting of chaotic dynamical systems.…”

Section: Long Short-term Memory Nudgingmentioning

confidence: 99%

Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Pawar,

Ahmed,

San

et al. 2020

Preprint

View full text Add to dashboard Cite

Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging gain matrix might be cumbersome. In this paper, we put forth a fully nonintrusive recurrent neural network approach based on a long short-term memory (LSTM) embedding architecture to estimate the nudging term, which plays a role not only to force the state trajectories to the observations but also acts as a stabilizer. Furthermore, our approach relies on the power of archival data and the trained model can be retrained effectively due to power of transfer learning in any neural network applications. In order to verify the feasibility of the proposed approach, we perform twin experiments using Lorenz 96 system. Our results demonstrate that the proposed LSTM nudging approach yields more accurate estimates than both extended Kalman filter (EKF) and ensemble Kalman filter (EnKF) when only sparse observations are available. With the availability of emerging AI-friendly and modular hardware technologies and heterogeneous computing platforms, we articulate that our simplistic nudging framework turns out to be computationally more efficient than either the EKF or EnKF approaches.

show abstract

Predicting dynamical system evolution with residual neural networks

Cited by 2 publications

References 0 publications

Tensorized Transformer for Dynamical Systems Modeling

Tensorized Transformer for Dynamical Systems Modeling

Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Contact Info

Product

Resources

About