Discovery of Dynamics Using Linear Multistep Methods

Abstract

Linear multistep methods (LMMs) are popular time discretization techniques for the numerical solution of differential equations. Traditionally they are applied to solve for the state given the dynamics (the forward problem), but here we consider their application for learning the dynamics given the state (the inverse problem). This repurposing of LMMs is largely motivated by growing interest in data-driven modeling of dynamics, but the behavior and analysis of LMMs for discovery turn out to be significantly different from the well-known, existing theory for the forward problem. Assuming the highly idealized setting of being given the exact state, we establish for the first time a rigorous framework based on refined notions of consistency and stability to yield convergence using LMMs for discovery. When applying these concepts to three popular M −step LMMs, the Adams-Bashforth, Adams-Moulton, and Backwards Differentiation Formula schemes, with M ∈ N, the new theory suggests that Adams-Bashforth for 1 ≤ M ≤ 6, Adams-Moulton for M = 0 and M = 1, and Backwards Differentiation Formula for all M are convergent, and, otherwise, the methods are not convergent in general. In addition, we provide numerical experiments to both motivate and substantiate our theoretical analysis.

Inverse modified differential equations for discovery of dynamics

“…It is proven that the discrepancy between f tag and f h depends on the order of the integrator. Similar results have been provided for specific integrators such as [24] for multistep methods. IMDEs illuminate the errors of discovery using general ODE solver from a new perspective.…”

“…To begin with, we check how neural ODEs act on the whole space and thus sample sufficient data to circumvent generalization problems. For a model problem, we consider the two-dimensional damped harmonic oscillator with cubic dynamics, which is also investigated in [24,28]. The equation is of the form…”

Weak SINDy: Galerkin-Based Data-Driven Model Selection

“…The authors of [14] used compressed sensing techniques to enforce sparsity. Since then there has been an explosion of interest in the problem of identifying nonlinear dynamical systems from data, with some of the primary techniques being Gaussian process regression [10], deep neural networks [11], Bayesian inference [18,19] and a variety of methods from numerical analysis [6,7]. These techniques have been successfully applied to discovery of both ordrinary and partial differential equations.…”

Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations

“…Another group of articles that are closely related to ours focus on applying neural networks to recover the dynamic system (the expression of F(•) in (1.1)) from observed data [39,40,41,42,43,44,45], in which points values data on one trajectory x(t) are collected and used in training. Classical numerical ODE solvers such as Runge-Kutta methods or Adam-version multistep methods are used in the definition of loss function to train the feed forward neural networks.…”