Bayesian estimation of the observation‐error covariance matrix in ensemble‐based filters

Abstract

We develop a Bayesian technique for estimating the parameters in the observation-noise covariance matrix R t for ensemble data assimilation. We design a posterior distribution by using the ensemble-approximated likelihood and a Wishart prior distribution and present an iterative algorithm for parameter estimation. The temporal smoothness of R t can be controlled by an adequate choice of two parameters of the prior distribution, the covariance matrix S and the number of degrees of freedom ν. The ν parameter can be estimated by maximizing the marginal likelihood. The present formalism can handle cases in which the number of data points or data positions varies with time, the former of which is exemplified in the experiments. We present an application to a coupled atmosphere-ocean model under each of the following assumptions: R t is a scalar multiple of a fixed matrix (R t = α t , where α t is the scalar parameter and is the fixed matrix), R t is diagonal, R t has fixed eigenvectors or R t has no specific structure. We verify that the proposed algorithm works well and that only a limited number of iterations are necessary. When R t has one of the structures mentioned above, by assuming S to be the previous estimate we obtain a Bayesian estimate of R t that varies smoothly in time compared with the maximum-likelihood estimate. When R t has no specific structure, we need to regularize S to maintain the positive-definiteness. Through twin experiments, we find that the best estimate of R t is, in general, obtained by a combination of structure-free R t and tapered S using decorrelation lengths of half the size of the model ocean basin. From experiments using real observations, we find that the estimates of the structured R t lead to overfitting of the data compared with the structure-free R t .

Uncertainty quantification of pollutant source retrieval: comparison of Bayesian methods with application to the Chernobyl and Fukushima Daiichi accidental releases of radionuclides

“…where x (i) n ∼ x|y, θ (i) . In the geosciences, this approach has been put forward by Ueno and Nakamura (2014), Tandeo et al (2015) and Ueno and Nakamura (2016), where observation, background and model error covariance matrices are estimated. Their physical…”

Estimating model‐error covariances in nonlinear state‐space models using Kalman smoothing and the expectation–maximization algorithm

“…(Ueno and Nakamura, 2014) used the EM algorithm to sequentially compute an approximation of the maximum likelihood estimate of the observation-error covariance in a data assimilation scheme using the EnKF. More recently, (Ueno and Nakamura, 2016) applied the EM algorithm for online estimation of the observation-error covariance matrix in a Bayesian framework.…”

Technical note: Assessment of observation quality for data assimilation in flood models

“…Instead observation uncertainties must be estimated statistically (e.g. Hollingsworth and Lönnberg, 1986;Ueno and Nakamura, 2016). Desroziers et al (2005) provide a diagnostic to estimate observation uncertainties using the statistical average of observation-minusbackground and observation-minus-analysis residuals.…”