SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives

Abstract

Gaussian processes are an important regression tool with excellent analytic properties which allow for direct integration of derivative observations. However, vanilla GP methods scale cubically in the amount of observations. In this work, we propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We then prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior. To furthermore illustrate the practical applicability of our method, we then apply it to ODIN, a recently developed algorithm for ODE parameter inference. In an extensive experiments section, all results are empirically validated, demonstrating the speed, accuracy, and practical applicability of this approach.

Distributional Gradient Matching for Learning Uncertain Neural Dynamics Models

“…In its original form, calculating the matrix inverses of both Equation ( 17) and Equation ( 18) has cubic complexity in N . To alleviate this problem, we follow Rahimi et al (2007) and Angelis et al (2020) by using a feature approximation of the kernel matrix and its derivatives. In particular, let Φ ∈ R F ×N be a matrix of F random Fourier features as described by Rahimi et al (2007).…”

“…Furthermore, denote Φ as its derivative w.r.t. the time input variable, as defined by Angelis et al (2020). We can now approximate the kernel matrix and its derivative versions as…”