Augmenting physical models with deep networks for complex dynamics forecasting*

Abstract

Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling-based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framework, a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists of decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model. The learning problem is carefully formulated such that the physical model explains as much of the data as possible, while the data-driven component only describes information that cannot be captured by the physical model; no more, no less. This not only provides the existence and uniqueness for this decomposition, but also ensures interpretability and benefit generalization. Experiments made on three important use cases, each representative of a different family of phenomena, i.e. reaction–diffusion equations, wave equations and the non-linear damped pendulum, show that APHYNITY can efficiently leverage approximate physical models to accurately forecast the evolution of the system and correctly identify relevant physical parameters. The code is available at https://github.com/yuan-yin/APHYNITY.

Robust Hybrid Learning With Expert Augmentation

“…Learning a model that is close to the expert model and fits the training data well is a hard problem. However, the APHYNITY algorithm (Yin et al, 2021) and the Hybrid-VAE (Takeishi & Kalousis, 2021, HVAE) are two recent approaches that offer promising solutions to this problem. We now briefly describe these two methods and how they can be used to approximate the Bayes optimal predictor of (1).…”

“…The damped pendulum is often used as an example in the hybrid modelling literature (Yin et al, 2021;Takeishi & Kalousis, 2021). The system's state at time t is y t = θ t θt T , where θ t is the angle of the pendulum at time t and θt its angular speed.…”

“…The 2D reaction diffusion was used by Yin et al (2021) to assess the quality of APHYNITY. It is a 2D FitzHugh-Nagumo on a 32 × 32 grid.…”

“…As an example, Neural ODE (Chen et al, 2018) and convolutional neural networks (LeCun et al, 1995, CNN) are privileged architectures to work with dynamical systems described by ODEs or PDEs. While most of the HyL's literature focus on the predictive performance of hybrid models, recent work have also showed that HyL may help to infer the physical parameter accurately (Yin et al, 2021;Takeishi & Kalousis, 2021). This is aligned with Zyla et al (2020) (see Section 40.2.2.2) which observe that inference on incomplete models results in a systematic bias.…”

“…In recent work (Yin et al, 2021;Takeishi & Kalousis, 2021;Qian et al, 2021;Mehta et al, 2020;Lei & Mirams, 2021;Reichstein et al, 2019), HyL demonstrated success in complementing partial physical models and improving the inference of the corresponding parameters. However, contrarily to the common belief that HyL achieves better generalization than black box ML models, we argue that hybrid models do not meet their promise regarding robustness.…”