Sparse Spatio-temporal Gaussian Processes with General Likelihoods

Abstract

Abstract. In this paper, we consider learning of spatio-temporal processes by formulating a Gaussian process model as a solution to an evolution type stochastic partial differential equation. Our approach is based on converting the stochastic infinite-dimensional differential equation into a finite dimensional linear time invariant (LTI) stochastic differential equation (SDE) by discretizing the process spatially. The LTI SDE is time-discretized analytically, resulting in a state space model with linear-Gaussian dynamics. We use expectation propagation to perform approximate inference on non-Gaussian data, and show how to incorporate sparse approximations to further reduce the computational complexity. We briefly illustrate the proposed methodology with a simulation study and with a real world modelling problem.

Rao-Blackwellized particle mcmc for parameter estimation in spatio-temporal Gaussian processes

“…to ∆x [6]. Furthermore, from (9) it is clear that the order of the linear system directly depends on the hyperparameter νt: Whenever p = νt − 1/2 is a positive integer, the following companion form state-space representation can be obtained…”

“…These approaches can also readily be employed when using the equivalent state-space formulations and Kalman filtering [7]. When facing non-Gaussian likelihoods p(yn | fn, θy), one commonly has to resort to approximations such as expectation propagation or Laplace approximations [9][10][11]. These may overcome some of the difficulties involved with nonlinear non-Gaussian likelihoods, but introduce approximations to the posterior and may still be unfeasible for a large number of data points and require further approximations such as inducing points [12].…”

Gaussian quadratures for state space approximation of scale mixtures of squared exponential covariance functions

“…, N , denotes the coordinate of the kth interval and y k the number of incidents in the interval. As the model is now non-Gaussian we have to resort to the Laplace approximation (finding the mode by a Newton scheme and forming a Gaussian approximation in the mode, see [1,10]) for doing the updates inside the Kalman filter. The data 1 contain the dates of 191 coal mine explosions that killed ten or more men in Britain between years 1851 and 1962, where the RQ assumptions of smoothness and longrange correlations are justified.…”

“…Furthermore, a complementary approach for converting periodic covariance functions to state space models was recently proposed in [4]. The approach also extends to spatio-temporal modeling (see, e.g., [3,8,9]) and non-Gaussian likelihoods (see [10] and Sec. 4.2).…”

On convergence and accuracy of state-space approximations of squared exponential covariance functions

“…It turns out [2][3][4] that the state-inference problem can be solved efficiently using classical Kalman filtering and smoothing [11][12][13][14], which has linear complexity O(N ) with respect to the number of measurements. The state-space GP methods have also been extended to non-Gaussian and non-linear settings and they have found many applications in location sensing, physics, and medicine [4,[15][16][17][18]. The main challenge in the state-space approach is the construction of the state-space model from a given covariance function prescription.…”