Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes

Abstract

Parameter estimation for nonlinear dynamic system models, represented by ordinary differential equations (ODEs), using noisy and sparse data, is a vital task in many fields. We propose a fast and accurate method, manifold-constrained Gaussian process inference (MAGI), for this task. MAGI uses a Gaussian process model over time series data, explicitly conditioned on the manifold constraint that derivatives of the Gaussian process must satisfy the ODE system. By doing so, we completely bypass the need for numerical integration and achieve substantial savings in computational time. MAGI is also suitable for inference with unobserved system components, which often occur in real experiments. MAGI is distinct from existing approaches as we provide a principled statistical construction under a Bayesian framework, which incorporates the ODE system through the manifold constraint. We demonstrate the accuracy and speed of MAGI using realistic examples based on physical experiments.

Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning

“…Gaussian Processes have been used to identify SDE from observations [52], with a target likelihood based on the Euler-Maruyama scheme (not explicitly mentioned by the authors). Our setting is very different from identifying ODE from noisy observations, for example through Gaussian Processes [51]. Generative stochastic modeling of strongly nonlinear flows with non-Gaussian statistics is also possible [3].…”

“…Minimizing L in (5) over the data D implies maximization of the log marginal likelihood (4) with the constant terms removed (as they do not influence the minimization) [41]. Likelihood estimation in combination with the normal distribution is used in many variational and generative approaches [19,28,33,52,51,49]. Note that here, the step size h (k) is defined per snapshot, so it is possible that it has different values for every index k. This will be especially useful in coarse-graining Gillespie simulations, where the time step is determined as part of the scheme.…”

InfPolyn, a Nonparametric Bayesian Characterization for Composition-Dependent Interdiffusion Coefficients

“…The explicit derivative information is further utilized to improve a general GP's performance for data that are generated from differential equations [24]. The derivative in a given system of differential equations is further harnessed through a constraint manifold such that the derivatives of the Gaussian process must match an ordinary differential Equation (ODE) [25]. Despite their success, these works generally require explicitly known differential equations to work.…”

“…A closely related work is [26], where GP is used as a generalization for a parametric function for binary images. However, their work cannot be directly implemented in our problem because our systems of equations will lead to a mixture of GPs that are augmented by the derivative of concentrations, whereas there is normally only one GP to estimate in most of the previous works [23][24][25][26][27].…”

Learning Interaction Kernels for Agent Systems on Riemannian Manifolds

“…This has motivated a growing body of research considering interacting agent systems on various manifolds [19,6,30], including opinion dynamics [2], flocking models [1] and a classical aggregation model [5]. Further recent approaches for interacting agents on manifolds include [37,32].…”