Bayesian Neural Ordinary Differential Equations

Abstract

Recently, Neural Ordinary Differential Equations has emerged as a powerful framework for modeling physical simulations without explicitly defining the ODEs governing the system, but learning them via machine learning. However, the question: "Can Bayesian learning frameworks be integrated with Neural ODE's to robustly quantify the uncertainty in the weights of a Neural ODE?" remains unanswered. In an effort to address this question, we demonstrate the successful integration of Neural ODEs with two methods of Bayesian Inference: (a) The No-U-Turn MCMC sampler (NUTS) and (b) Stochastic Langevin Gradient Descent (SGLD). We test the performance of our Bayesian Neural ODE approach on classical physical systems, as well as on standard machine learning datasets like MNIST, using GPU acceleration. Finally, considering a simple example, we demonstrate the probabilistic identification of model specification in partially-described dynamical systems using universal ordinary differential equations. Together, this gives a scientific machine learning tool for probabilistic estimation of epistemic uncertainties.

Learning orbital dynamics of binary black hole systems from gravitational wave measurements

“…The smaller orbiting object's motion is then described by a geodesic path in the Schwarzschild geometry set by the larger black hole. Recent numerical evidence suggests that the geodesic equations of motion (11) with self-force corrections may work unreasonably well even for near-equal mass systems [19][20][21][22][23][24][25][26][27]. More importantly for our purposes, we know from blackhole perturbation theory results that using geodesic equations of motion to describe the two-body problem neglects a substantial amount of physics including terms proportional to m 2 /m 1 .…”

“…(5) have been inspired by Eq. (11), which are the geodesic equations of motion for an infinitesimally small "particle" orbiting a super-massive blackhole. In particular, Eqs.…”

“…In Figure 2 we compare the true waveforms/trajectories to the learned waveforms/trajectories over the extended time interval [0, 3•10 5 ]. To compare the learned model ( 5) to the exact model (11), we also compute the error in φ and χ over the extended time interval. Evidently, not only do the waveforms and trajectories match upon visual inspection, the learned model matches the true mechanical model to about two orders of magnitude.…”

“…We also show the portion of the orbit (black) corresponding to the gravitational-wave training window, although no orbital data was used to learn the dynamics. Top right: Relative error between the learned model ( 5) and the exact model (11). Bottom: Learned (dashed red) and exact (solid blue) waveforms extrapolated 4× the training interval.…”

“…In this paper, we follow a different approach to learning physical equations: we solve an optimization problem which isolates the most likely physical model (differential equations) that would deliver certain physical measurements (data). This approach is aligned with a growing trend in data-driven science; see, e.g., [1][2][3][4][5][6][7][8][9][10][11] and references therein.…”