A probabilistic model for the numerical solution of initial value problems

Abstract

We study connections between ordinary differential equation (ODE) solvers and probabilistic regression methods in statistics. We provide a new view of probabilistic ODE solvers as active inference agents operating on stochastic differential equation models that estimate the unknown initial value problem (IVP) solution from approximate observations of the solution derivative, as provided by the ODE dynamics. Adding to this picture, we show that several multistep methods of Nordsieck form can be recasted as Kalman filtering on q-times integrated Wiener processes. Doing so provides a family of IVP solvers that return a Gaussian posterior measure, rather than a point estimate. We show that some such methods have low computational overhead, nontrivial convergence order, and that the posterior has a calibrated concentration rate. Additionally, we suggest a step size adaptation algorithm which completes the proposed method to a practically useful implementation, which we experimentally evaluate using a representative set of standard codes in the DETEST benchmark set.

Probabilistic solutions to ordinary differential equations as nonlinear Bayesian filtering: a new perspective

“…Hence the extended Kalman filter computes the residual, z n −ẑ n , in the same manner as Schober et al (2019). However, as the filter gain, K n , now depends on evaluations of the Jacobian, the resulting probabilistic ODE solver is different in general.…”

“…where X (1) (t) and X (2) (t) model y(t) andẏ(t), respectively. The remaining q − 1 sub-vectors in X(t) can be used to model higher order derivatives of y(t) as done by Schober et al (2019) and Kersting and Hennig (2016). We define such priors by a stochastic differential equation (Øksendal, 2003), that is,…”

“…Inference requires data, and an associated likelihood. Previous authors, such as Schober et al (2019) and Chkrebtii et al (2016), put forth the view of the prior measure defining an inference agent, which cycles through extrapolating, generating measurements of the vector field, and updating. Here we argue that there is no need for generating measurements, since re-writing Eq.…”

“…For any fixed-time discretisation, the measurement sequence and likelihood are also fixed. That is, we avoid the coupling of prior and measurement sequence, that is for example present in Schober et al (2019). This enables application of all Bayesian filtering and smoothing techniques to ODEs as described, for example, in Särkkä (2013).…”

A modern retrospective on probabilistic numerics

“…For example, in a quadrature setting, the practitioner is in the fortunate position of being able to use evaluations of the integrand u both to estimate the regularity of u and the value of the integral. Empirical Bayesian methods are explored in [Schober et al, 2018] and in [Jagadeeswaran and Hickernell, 2018].…”