Bayesian Filtering for ODEs with Bounded Derivatives

Abstract

Recently there has been increasing interest in probabilistic solvers for ordinary differential equations (ODEs) that return full probability measures, instead of point estimates, over the solution and can incorporate uncertainty over the ODE at hand, e.g. if the vector field or the initial value is only approximately known or evaluable. The ODE filter proposed in [9, 16] models the solution of the ODE by a Gauss-Markov process which serves as a prior in the sense of Bayesian statistics. While previous work employed a Wiener process prior on the (possibly multiple times) differentiated solution of the ODE and established equivalence of the corresponding solver with classical numerical methods, this paper raises the question whether other priors also yield practically useful solvers. To this end, we discuss a range of possible priors which enable fast filtering and propose a new prior-the Integrated Ornstein Uhlenbeck Process (IOUP)-that complements the existing Integrated Wiener process (IWP) filter by encoding the property that a derivative in time of the solution is bounded in the sense that it tends to drift back to zero. We provide experiments comparing IWP and IOUP filters which support the belief that IWP approximates better divergent ODE's solutions whereas IOUP is a better prior for trajectories with bounded derivatives.

Convergence Rates of Gaussian ODE Filters

“…after the sequence of error covariance matrices has converged. These results neither cover filters with the integrated Ornstein-Uhlenbeck process (IOUP) prior [20] nor non-zero noise models on evaluations of f .…”

“…A global analysis for IOUP is therefore more complicated than for IBM: Recall from (2.6) that, for q = 1, the mean prediction for x((n + 1)h) is m −,(0) ((n + 1)h) m −, (1) (1) (nh) , (5.5) which pulls both m −,(0) and m −, (1) towards zero (or some other prior mean) compared to its Taylor-expansion prediction for θ = 0. While this is useful for ODEs converging to zero, such as ẋ = −x, it is problematic for diverging ODEs, such as ẋ = x [20]. As shown in Theorem 5.2, this effect is asymptotically negligible for local convergence, but it might matter globally and, therefore, might necessitate stronger assumptions on f than Assumption 1, such as a bound on f ∞ which would globally bound {y(nh); n = 0, .…”

“…In this paper, we determine the convergence rates of a recent family of PN methods [34,15,20,35,41] which recast an IVP as a stochastic filtering problem [26,Chapter 6], an approach that has been studied in other settings [13], but has not been applied to IVPs before. These methods assume a priori that the solution x and its first q ∈ N derivatives follow a Gauss-Markov process X that solves a stochastic differential equation (SDE).…”

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

“…The former class includes classical ODE solvers that are stochastically perturbed (Abdulle and Garegnani, 2020, Conrad et al, 2017, Lie et al, 2019, Teymur et al, 2018, solvers that approximately sample from a Bayesian inference problem (Tronarp et al, 2019b), and solvers that perform Gaussian process regression on stochastically generated data (Chkrebtii et al, 2016). On the other hand, deterministic solvers formulate the problem as a Gaussian process regression problem, either with a data generation mechanism (Hennig and Hauberg, 2014, Kersting and Hennig, 2016, Magnani et al, 2017, Schober et al, 2014, Skilling, 1992 or by attempting to constrain the estimate to solve the ODE at each point on the mesh (John et al, 2019, Tronarp et al, 2019b. For computational reasons it is fruitful to select the Gaussian process prior to be of Markov type (Kersting and Hennig, 2016, Magnani et al, 2017, Tronarp et al, 2019b, as this reduces cost of inference from O(N 3 ) to O(N ) Särkkä, 2010, Särkkä et al, 2013).…”

“…On the other hand, deterministic solvers formulate the problem as a Gaussian process regression problem, either with a data generation mechanism (Hennig and Hauberg, 2014, Kersting and Hennig, 2016, Magnani et al, 2017, Schober et al, 2014, Skilling, 1992 or by attempting to constrain the estimate to solve the ODE at each point on the mesh (John et al, 2019, Tronarp et al, 2019b. For computational reasons it is fruitful to select the Gaussian process prior to be of Markov type (Kersting and Hennig, 2016, Magnani et al, 2017, Tronarp et al, 2019b, as this reduces cost of inference from O(N 3 ) to O(N ) Särkkä, 2010, Särkkä et al, 2013). Because of the connection between inference with Gauss-Markov processes priors and spline interpolation (Kimeldorf and Wahba, 1970, Sidhu and Weinert, 1979, Weinert and Kailath, 1974, the Gaussian process regression approaches are intimately connected with the spline approach to ODEs (Schumaker, 1982, Wahba, 1973.…”