Differentiable Likelihoods for Fast Inversion of 'Likelihood-Free' Dynamical Systems

Abstract

Likelihood-free (a.k.a. simulation-based) inference problems are inverse problems with expensive, or intractable, forward models. ODE inverse problems are commonly treated as likelihood-free, as their forward map has to be numerically approximated by an ODE solver. This, however, is not a fundamental constraint but just a lack of functionality in classic ODE solvers, which do not return a likelihood but a point estimate. To address this shortcoming, we employ Gaussian ODE filtering (a probabilistic numerical method for ODEs) to construct a local Gaussian approximation to the likelihood. This approximation yields tractable estimators for the gradient and Hessian of the (log-) likelihood. Insertion of these estimators into existing gradient-based optimization and sampling methods engenders new solvers for ODE inverse problems. We demonstrate that these methods outperform standard likelihood-free approaches on three benchmark-systems.

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

“…Classically, the error of a numerical solver has been quantified in terms of the worst case error. However, in applications where a numerical solution is sought as a component of a larger statistical inference problem (see, e.g., Matsuda andMiyatake 2019 andKersting et al 2020), it is desirable that the error can be quantified with the same semantic, that is to say, probabilistically (Hennig et al, 2015, Oates and. Hence the recent endeavour to develop probabilistic ODE solvers.…”