Representation, representativity, representativeness error, forward interpolation error, forward model error, observation-operator error, aggregation error and sampling error are all terms used to refer to components of observation error in the context of data assimilation. This article is an attempt to consolidate the terminology that has been used in the earth sciences literature and was suggested at a European Space Agency workshop held in Reading in April 2014. We review the state of the art and, through examples, motivate the terminology. In addition to a theoretical framework, examples from application areas of satellite data assimilation, ocean reanalysis and atmospheric chemistry data assimilation are provided. Diagnosing representation-error statistics as well as their use in state-of-the-art data assimilation systems is discussed within a consistent framework.

Abstract.We have developed an ensemble Kalman Filter (EnKF) to estimate 8-day regional surface fluxes of CO 2 from space-borne CO 2 dry-air mole fraction observations (X CO 2 ) and evaluate the approach using a series of synthetic experiments, in preparation for data from the NASA Orbiting Carbon Observatory (OCO). The 32-day duty cycle of OCO alternates every 16 days between nadir and glint measurements of backscattered solar radiation at short-wave infrared wavelengths. The EnKF uses an ensemble of states to represent the error covariances to estimate 8-day CO 2 surface fluxes over 144 geographical regions. We use a 12×8-day lag window, recognising that X CO 2 measurements include surface flux information from prior time windows. The observation operator that relates surface CO 2 fluxes to atmospheric distributions of X CO 2 includes: a) the GEOS-Chem transport model that relates surface fluxes to global 3-D distributions of CO 2 concentrations, which are sampled at the time and location of OCO measurements that are cloud-free and have aerosol optical depths <0.3; and b) scene-dependent averaging kernels that relate the CO 2 profiles to X CO 2 , accounting for differences between nadir and glint measurements, and the associated scene-dependent observation errors. We show that OCO X CO 2 measurements significantly reduce the uncertainties of surface CO 2 flux estimates. Glint measurements are generally better at constraining ocean CO 2 flux estimates. Nadir X CO 2 measurements over the terrestrial tropics are sparse throughout the year because of either clouds or smoke. Glint measurements provide the most effective constraint for estimating tropical terrestrial CO 2 fluxes by accurately sampling fresh continental outflow over neighbouring oceans. We also present results from sensitivity experiments that investigate how flux estimates change with 1) bias and Correspondence to: L. Feng (lfeng@staffmail.ed.ac.uk) unbiased errors, 2) alternative duty cycles, 3) measurement density and correlations, 4) the spatial resolution of estimated flux estimates, and 5) reducing the length of the lag window and the size of the ensemble. At the revision stage of this manuscript, the OCO instrument failed to reach its orbit after it was launched on 24 February 2009. The EnKF formulation presented here is also applicable to GOSAT measurements of CO 2 and CH 4 .

Data assimilation provides techniques for combining observations and prior model forecasts to create initial conditions for numerical weather prediction (NWP). The relative weighting assigned to each observation in the analysis is determined by its associated error. Remote sensing data usually has correlated errors, but the correlations are typically ignored in NWP. Here we describe three approaches to the treatment of observation error correlations. For an idealised data set, the information content under each simplified assumption is compared to that under the correct correlation specification. Treating the errors as uncorrelated results in a significant loss of information. However, retention of an approximated correlation gives clear benefits.

A B S T R A C T Remote sensing observations often have correlated errors, but the correlations are typically ignored in data assimilation for numerical weather prediction. The assumption of zero correlations is often used with data thinning methods, resulting in a loss of information. As operational centres move towards higher-resolution forecasting, there is a requirement to retain data providing detail on appropriate scales. Thus an alternative approach to dealing with observation error correlations is needed. In this article, we consider several approaches to approximating observation error correlation matrices: diagonal approximations, eigendecomposition approximations and Markov matrices. These approximations are applied in incremental variational assimilation experiments with a 1-D shallow water model using synthetic observations. Our experiments quantify analysis accuracy in comparison with a reference or 'truth' trajectory, as well as with analyses using the 'true' observation error covariance matrix. We show that it is often better to include an approximate correlation structure in the observation error covariance matrix than to incorrectly assume error independence. Furthermore, by choosing a suitable matrix approximation, it is feasible and computationally cheap to include error correlation structure in a variational data assimilation algorithm.

SummaryIn variational data assimilation a least-squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least-squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least-squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian.We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.

This article is published with the permission of the Controller of HMSO and the Queen's Printer for Scotland.The optimal utilisation of hyper-spectral satellite observations in numerical weather prediction is often inhibited by incorrectly assuming independent interchannel observation errors. However, in order to represent these observation-error covariance structures, an accurate knowledge of the true variances and correlations is needed. This structure is likely to vary with observation type and assimilation system. The work in this article presents the initial results for the estimation of IASI interchannel observation-error correlations when the data are processed in the Met Office one-dimensional (1D-Var) and four-dimensional (4D-Var) variational assimilation systems. The method used to calculate the observation errors is a post-analysis diagnostic which utilises the background and analysis departures from the two systems.The results show significant differences in the source and structure of the observation errors when processed in the two different assimilation systems, but also highlight some common features. When the observations are processed in 1D-Var, the diagnosed error variances are approximately half the size of the error variances used in the current operational system and are very close in size to the instrument noise, suggesting that this is the main source of error. The errors contain no consistent correlations, with the exception of a handful of spectrally close channels. When the observations are processed in 4D-Var, we again find that the observation errors are being overestimated operationally, but the overestimation is significantly larger for many channels. In contrast to 1D-Var, the diagnosed error variances are often larger than the instrument noise in 4D-Var. It is postulated that horizontal errors of representation, not seen in 1D-Var, are a significant contributor to the overall error here. Finally, observation errors diagnosed from 4D-Var are found to contain strong, consistent correlation structures for channels sensitive to water vapour and surface properties.

With the development of convection-permitting numerical weather prediction the efficient use of high-resolution observations in data assimilation is becoming increasingly important. The operational assimilation of these observations, such as Doppler radar radial winds (DRWs), is now common, although to avoid violating the assumption of uncorrelated observation errors the observation density is severely reduced. To improve the quantity of observations used and the impact that they have on the forecast requires the introduction of the full, potentially correlated, error statistics. In this work, observation error statistics are calculated for the DRWs that are assimilated into the Met Office high-resolution U.K. model (UKV) using a diagnostic that makes use of statistical averages of observation-minus-background and observation-minus-analysis residuals. This is the first in-depth study using the diagnostic to estimate both horizontal and along-beam observation error statistics. The new results obtained show that the DRW error standard deviations are similar to those used operationally and increase as the observation height increases. Surprisingly, the estimated observation error correlation length scales are longer than the operational thinning distance. They are dependent both on the height of the observation and on the distance of the observation away from the radar. Further tests show that the long correlations cannot be attributed to the background error covariance matrix used in the assimilation, although they are, in part, a result of using superobservations and a simplified observation operator. The inclusion of correlated error statistics in the assimilation allows less thinning of the data and hence better use of the high-resolution observations.

No abstract