On the representation error in data assimilation

Janjić, Tijana; Bormann, Niels; Bocquet, Marc; Carton, James A.; Cohn, Stephen E.; Dance, Sarah L.; Losa, Svetlana N.; Nichols, Nancy K.; Potthast, Roland; Waller, John C.; Weston, Peter

doi:10.1002/qj.3130

Cited by 249 publications

(283 citation statements)

References 124 publications

Supporting

Mentioning

283

Contrasting

Order By: Relevance

“…1,2 One of the most well-known applications of data assimilation is to numerical weather prediction (NWP), where observations of the atmosphere and ocean are combined with a prior model state of the atmosphere in order to produce the initial conditions for a weather forecast. Until recently, diagonal observation error covariance matrices have been used operationally at all major NWP centres, 3 a choice that is only valid in the case that observation errors are uncorrelated.…”

Section: Discussionmentioning

confidence: 99%

The conditioning of least‐squares problems in variational data assimilation

Tabeart

Dance

Haben

et al. 2018

Numerical Linear Algebra App

Self Cite

103

View full text Add to dashboard Cite

SummaryIn variational data assimilation a least-squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least-squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least-squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian.We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.

show abstract

Section: Discussionmentioning

confidence: 99%

The conditioning of least‐squares problems in variational data assimilation

Tabeart

Dance

Haben

et al. 2018

Numerical Linear Algebra App

Self Cite

103

View full text Add to dashboard Cite

show abstract

“…The method could, in principle, provide guidance for any assimilation system. By considering the observation space subdomain [25], proper scaling, local averaging [26], or other methods discussed in Janjic et al [12] it may also be possible to extend this methodology to spatially varying error statistics. Based on our verification results in Part I [5], we found that there is a dependence between model values and error variances, which we will investigate further in view of our next operational implementation of the Canadian surface air quality analysis and assimilation.…”

Section: Discussionmentioning

confidence: 99%

“…Although this assumption is never entirely observed in reality, there are ways to work around it. In the case of in situ observations, and assuming that any systematic error have been removed, random errors are still present, due to the difference between the observation and the model's equivalent of the observation-called representativeness error (see Janjic et al [12] for a review). Representativeness error is due to unresolved scales and processes in the model and interpolation or forward observation The mean perceived analysis error variance for all experiments is presented in Table 4.…”

Section: Representativeness Error With In Situ Observationsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Analysis by Cross-Validation, Part II: Diagnostic and Optimization of Analysis Error Covariance

Ménard

Deshaies-Jacques

2018

Atmosphere

View full text Add to dashboard Cite

Abstract:We present a general theory of estimation of analysis error covariances based on cross-validation as well as a geometric interpretation of the method. In particular, we use the variance of passive observation-minus-analysis residuals and show that the true analysis error variance can be estimated, without relying on the optimality assumption. This approach is used to obtain near optimal analyses that are then used to evaluate the air quality analysis error using several different methods at active and passive observation sites. We compare the estimates according to the method of Hollingsworth-Lönnberg, Desroziers et al., a new diagnostic we developed, and the perceived analysis error computed from the analysis scheme, to conclude that, as long as the analysis is near optimal, all estimates agree within a certain error margin.

show abstract

“…ensemble Kalman filters), an over-reduced posterior covariance and unstable long-term forecast/assimilation cycles. Thus, to reduce the number of correlated observations and to avoid dealing with the spatial correlation in the assimilation, the current approach is to further thin the data (as is standard in other assimilation applications such as NWP and oceanography; Dando et al, 2007;Li et al, 2010). The applied thinning, as described in Mason et al (2012a), uses a top down clustering approach in which principal component analysis is used to select observations that have the highest information content.…”

Section: Quality Control and Data Thinningmentioning

confidence: 99%

Technical note: Assessment of observation quality for data assimilation in flood models

Waller

García-Pintado

Mason

et al. 2018

Hydrol. Earth Syst. Sci.

Self Cite

View full text Add to dashboard Cite

Abstract. The assimilation of satellite-based water level observations (WLOs) into 2-D hydrodynamic models can keep flood forecasts on track or be used for reanalysis to obtain improved assessments of previous flood footprints. In either case, satellites provide spatially dense observation fields, but with spatially correlated errors. To date, assimilation methods in flood forecasting either incorrectly neglect the spatial correlation in the observation errors or, in the best of cases, deal with it by thinning methods. These thinning methods result in a sparse set of observations whose error correlations are assumed to be negligible. Here, with a case study, we show that the assimilation diagnostics that make use of statistical averages of observation-minus-background and observation-minus-analysis residuals are useful to estimate error correlations in WLOs. The average estimated correlation length scale of 7 km is longer than the expected value of 250 m. Furthermore, the correlations do not decrease monotonically; this unexpected behaviour is shown to be the result of assimilating some anomalous observations. Accurate estimates of the observation error statistics can be used to support quality control protocols and provide insight into which observations it is most beneficial to assimilate. Therefore, the understanding gained in this paper will contribute towards the correct assimilation of denser datasets.

show abstract

On the representation error in data assimilation

Cited by 249 publications

References 124 publications

The conditioning of least‐squares problems in variational data assimilation

The conditioning of least‐squares problems in variational data assimilation

Evaluation of Analysis by Cross-Validation, Part II: Diagnostic and Optimization of Analysis Error Covariance

Technical note: Assessment of observation quality for data assimilation in flood models

Contact Info

Product

Resources

About