Maximum-Likelihood Estimation of Forecast and Observation Error Covariance Parameters. Part I: Methodology

Dee, Dick; Silva, Arlindo da

doi:10.1175/1520-0493(1999)127<1822:mleofa>2.0.co;2

Cited by 147 publications

(118 citation statements)

References 26 publications

Supporting

Mentioning

117

Contrasting

Order By: Relevance

“…For a discussion of covariance estimation techniques in the context of NWP, (see, e.g. [28,41,15,16]). Fortunately, the simulation results of Section 6 illustrate that the improvement in MSE due to tapering is robust to the choice of taper function and of taper length.…”

Section: Summary and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants

Furrer

Bengtsson²

2007

Journal of Multivariate Analysis

334

332

View full text Add to dashboard Cite

This work studies the effects of sampling variability in Monte Carlo-based methods to estimate very highdimensional systems. Recent focus in the geosciences has been on representing the atmospheric state using a probability density function, and, for extremely high-dimensional systems, various sample-based Kalman filter techniques have been developed to address the problem of real-time assimilation of system information and observations. As the employed sample sizes are typically several orders of magnitude smaller than the system dimension, such sampling techniques inevitably induce considerable variability into the state estimate, primarily through prior and posterior sample covariance matrices. In this article, we quantify this variability with mean squared error measures for two Monte Carlo-based Kalman filter variants: the ensemble Kalman filter and the ensemble square-root Kalman filter. Expressions of the error measures are derived under weak assumptions and show that sample sizes need to grow proportionally to the square of the system dimension for bounded error growth. To reduce necessary ensemble size requirements and to address rank-deficient sample covariances, covariance-shrinking (tapering) based on the Schur product of the prior sample covariance and a positive definite function is demonstrated to be a simple, computationally feasible, and very effective technique. Rules for obtaining optimal taper functions for both stationary as well as non-stationary covariances are given, and optimal taper lengths are given in terms of the ensemble size and practical range of the forecast covariance. Results are also presented for optimal covariance inflation. The theory is verified and illustrated with extensive simulations.

show abstract

Section: Summary and Discussionmentioning

confidence: 99%

“…Evaluating the expectation of the last term in the previous equation all the terms not involving 1/n 2 cancel. Combining (39) and (40) gives the third order approximation given in (14) and (15).…”

Section: Summary and Discussionmentioning

confidence: 99%

Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants

Furrer

Bengtsson²

2007

Journal of Multivariate Analysis

334

332

View full text Add to dashboard Cite

show abstract

“…This empirical algorithm has been adapted to retrieve AOT directly from cloud-cleared MODIS reflectances. Online quality control is performed with the adaptive buddy check of Dee et al (2001), with observation and background errors estimated using the maximum likelihood approach of Dee and da Silva (1999). Following a multi-channel AOT analysis, three-dimensional analysis increments are produced exploring the Lagrangian characteristics of the problem, generating local displacement ensembles intended to represent misplacements of the aerosol plumes.…”

Section: A14 Nasa Geos-5mentioning

confidence: 99%

Development towards a global operational aerosol consensus: basic climatological characteristics of the International Cooperative for Aerosol Prediction Multi-Model Ensemble (ICAP-MME)

Reid²,

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Too small or too large an inflation factor will cause the analysis state to over rely on the model forecasts or observations and can seriously undermine the accuracy and stability of the filter. In later studies, the inflation factor is estimated online based on the observation-minus-forecast residual (innovation statistic) [16,17] with different conditions. Past work shows that moment estimation can facilitate the calculation by solving an equation of the observation-minus-forecast residual and its realization [18][19][20].…”

Section: Introductionmentioning

confidence: 99%

The Error Covariance Matrix Inflation in Ensemble Kalman Filter

Wu¹,

Zheng²

2018

Kalman Filters - Theory for Advanced Applications

View full text Add to dashboard Cite

The estimation accuracy of ensemble forecast errors is crucial to the assimilation results for all ensemble-based schemes. The ensemble Kalman filter (EnKF) is a widely used scheme in land surface data assimilation, without using the adjoint of a dynamical model. In EnKF, the forecast error covariance matrix is estimated as the sampling covariance matrix of the ensemble forecast states. However, past researches on EnKF have found that it can generally lead to an underestimate of the forecast error covariance matrix, due to the limited ensemble size, as well as the poor initial perturbations and model error. This can eventually result in filter divergence. Therefore, using inflation to further adjust the forecast error covariance matrix becomes increasingly important. In this chapter, a new structure of the forecast error covariance matrix is proposed to mitigate the problems with limited ensemble size and model error. An adaptive procedure equipped with a second-order least squares method is applied to estimate the inflation factors of forecast and observational error covariance matrices. The proposed method is tested on the well-known atmosphere-like Lorenz-96 model with spatially correlated observational systems. The experiment results show that the new structure of the forecast error covariance matrix and the adaptive estimation procedure lead to improvement of the analysis states.

show abstract

Maximum-Likelihood Estimation of Forecast and Observation Error Covariance Parameters. Part I: Methodology

Cited by 147 publications

References 26 publications

Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants

Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants

Development towards a global operational aerosol consensus: basic climatological characteristics of the International Cooperative for Aerosol Prediction Multi-Model Ensemble (ICAP-MME)

The Error Covariance Matrix Inflation in Ensemble Kalman Filter

Contact Info

Product

Resources

About