Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions

Karvonen, Toni; Wynne, George G; Tronarp, Filip; Oates, Chris J.; Särkkä, Simo

doi:10.48550/arxiv.2001.10965

Cited by 2 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bounds for worst case overconfidence and underconfidence under maximum likelihood estimation of σ 2 has recently been obtained by Karvonen et al (2020). These results appear to carry over to the present setting for affine vector fields.…”

Section: Calibrating the Noise Scalesupporting

confidence: 59%

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

Tronarp¹,

Särkkä²,

Hennig³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

It has recently been established that the numerical solution of ordinary differential equations can be posed as a nonlinear Bayesian inference problem, which can be approximately solved via Gaussian filtering and smoothing, whenever a Gauss-Markov prior is used. In this paper the class of ν times differentiable linear time invariant Gauss-Markov priors is considered. A taxonomy of Gaussian estimators is established, with the maximum a posteriori estimate at the top of the hierarchy, which can be computed with the iterated extended Kalman smoother. The remaining three classes are termed explicit, semi-implicit, and implicit, which are in similarity with the classical notions corresponding to conditions on the vector field, under which the filter update produces a local maximum a posteriori estimate. The maximum a posteriori estimate corresponds to an optimal interpolant in the reproducing Hilbert space associated with the prior, which in the present case is equivalent to a Sobolev space of smoothness ν +1. Consequently, using methods from scattered data approximation and nonlinear analysis in Sobolev spaces, it is shown that the maximum a posteriori estimate converges to the true solution at a polynomial rate in the fill-distance (maximum step size) subject to mild conditions on the vector field. The methodology developed provides a novel and more natural approach to study the convergence of these estimators than classical methods of convergence analysis. The methods and theoretical results are demonstrated in numerical examples.

show abstract

Section: Calibrating the Noise Scalesupporting

confidence: 59%

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

Tronarp¹,

Särkkä²,

Hennig³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As a next step, it would be interesting to combine the results in this paper with results on the convergence of the estimated hyper-parameters θ N . For example, the recent work [20] studies the asymptotics of the maximum likelihood estimator of the marginal variance σ 2 in the Matérn model, under assumptions similar to this work. It would also be useful to include the Gaussian covariance kernel, corresponding to the limit ν = ∞ in the Matérn model, in our results.…”

Section: Conclusion and Discussionmentioning

confidence: 94%

Convergence of Gaussian Process Regression with Estimated Hyper-Parameters and Applications in Bayesian Inverse Problems

Teckentrup¹

2020

SIAM/ASA J. Uncertainty Quantification

View full text Add to dashboard Cite

This work is concerned with the convergence of Gaussian process regression. A particular focus is on hierarchical Gaussian process regression, where hyper-parameters appearing in the mean and covariance structure of the Gaussian process emulator are a-priori unknown, and are learnt from the data, along with the posterior mean and covariance. We work in the framework of empirical Bayes, where a point estimate of the hyper-parameters is computed, using the data, and then used within the standard Gaussian process prior to posterior update. We provide a convergence analysis that (i) holds for a given, deterministic function f to be emulated; and (ii) shows that convergence of Gaussian process regression is unaffected by the additional learning of hyper-parameters from data, and is guaranteed in a wide range of scenarios. As the primary motivation for the work is the use of Gaussian process regression to approximate the data likelihood in Bayesian inverse problems, we provide a bound on the error introduced in the Bayesian posterior distribution in this context.

show abstract

Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions

Cited by 2 publications

References 47 publications

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

Bayesian ODE Solvers: The Maximum A Posteriori Estimate

Convergence of Gaussian Process Regression with Estimated Hyper-Parameters and Applications in Bayesian Inverse Problems

Contact Info

Product

Resources

About