Physics-Informed Echo State Networks for Chaotic Systems Forecasting

Abstract

We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training of the ESNs, which penalizes non-physical predictions without the need of any additional training data. This approach is demonstrated on a chaotic Lorenz system, where the physics-informed ESNs improve the predictability horizon by about two Lyapunov times as compared to conventional ESNs. The proposed framework shows the potential of using machine learning combined with prior physical knowledge to improve the time-accurate prediction of chaotic dynamical systems.

Learning Ergodic Averages in Chaotic Systems

“…(1), results in or the ESN has a co-existing symmetric attractor. While this seemed not to have been an issue in short-term prediction, such as in [5], it does pose a problem in the long-term prediction of statistical quantities. This is because the ESN, in its present form, can not generate non-symmetric attractors.…”

“…The ESN's performance can be increased by incorporating physical knowledge during training [5] or during training and prediction [14]. This physical knowledge is usually present in the form of a reduced-order model (ROM) that can generate (imperfect) predictions.…”

“…This physical knowledge is usually present in the form of a reduced-order model (ROM) that can generate (imperfect) predictions. The authors of [5] introduced a physics-informed ESN (PI-ESN), which constrains the physics as a hard constraint with a physics loss term. The prediction is consistent with the physics, but the training requires nonlinear optimization.…”

“…Here, we use an hESN (Fig. 1) because we are not interested in constraining the physics as a hard constraint for an accurate shortterm prediction [5]. In the hESN, similarly to the conventional ESN, the input is fed to the reservoir via the input layer W in , but also to a physical model, which is usually a set of ordinary differential equations that approximately describe the system that is to be predicted.…”

“…These approaches, called physics-informed machine learning, have been applied to various problems in fluid dynamics [4,6]. For example, [5,14] improve the predictability horizon of echo state networks by leveraging physical knowledge, which is enforced as a hard constraint in [5], without needing more data or neurons. In this study, we use a hybrid echo state network (hESN) [14], originally proposed to time-accurately forecast the evolution of chaotic dynamical systems, to predict the long-term time averaged quantities, i.e., the ergodic averages.…”