Adaptive data assimilation including the effect of spatial variations in observation error

Abstract

SUMMARYAn optimal adaptive data assimilation algorithm is derived using the maximum likelihood method based on a conditional Gaussian probability density function for the first-guess and direct observations of the state variables but including local estimates of the observation and first-guess error statistics. An interpolation of the first-guess field to the observation coordinates is not required under the assumption of locally homogeneous statistics for the random atmosphere. However, the definition of observation error requires a definition of model 'truth' which is defined as a spatial average of the continuous random atmospheric variables. Then the total observation error consists of two independent components: an instrument error and an observation sampling error defined by the spatial average of the observation and the statistics of the local turbulence. Estimates of the observation sampling error statistics are determined from an ensemble of background or first-guess fields or from the analysis of the raw data from instrumented aircraft, Doppler lidars, or radar profilers. The spatial variations of the sampling error are referenced to the local turbulence conditions at each analysis coordinate and therefore each observation can have a different observation error for each nearby analysis coordinate. The extension of the adaptive assimilation concept to include the spatial variations in observation error for statistical interpolation, 3D-Var, 4D-Var, extended Kalman filtering, and ensemble Kalman filtering is also presented for the traditional meaning of observation error, i.e. each observation is assigned a single error. The conditional analysis error is derived for a single observation at the analysis coordinate and multiple observations around the analysis point. Example calculations of the conditional analysis error are presented for a few simple set of observation and measurement geometries to demonstrate the impact of the spatially variable observation errors. For rawinsonde observations, the observation sampling error is an important contribution to optimal adaptive data assimilation. Observations performed as a spatial average along a line, such as aircraft, radar profiler and Doppler lidar measurements, have smaller sampling errors than rawinsonde observations and therefore reduce the analysis error if the instrument error is sufficiently low.

Adjoint sensitivity of the model forecast to data assimilation system error covariance parameters

“…In practice, several simplifying assumptions are necessary to achieve a feasible implementation, and an increased amount of research in numerical weather prediction (NWP) is dedicated to observation-and background-error covariance modelling (Gaspari and Cohn, 1999;Hamill and Snyder, 2002;Lorenc, 2003;Buehner et al, 2005;Frehlich, 2006;Janjić and Cohn, 2006;Bannister, 2008a,b) and to the development of effective techniques for diagnosis, estimation, and tuning of unknown error covariance parameters in both variational and Kalman filter-based assimilation systems (Dee, 1995;Dee and Da Silva, 1999;Desroziers and Ivanov, 2001;Desroziers et al, 2005;Chapnik et al, 2006;Desroziers et al, 2009;Li et al, 2009).…”

The definition of ‘truth’ for Numerical Weather Prediction error statistics

“…Therefore, error should be defined in terms of 'truth', i.e. a spatial average of the continuous state variables based on the effective model filter (Frehlich, 2006). In addition, we assume that the model numerics and all sources of spatial filtering are universal, i.e.…”

“…The main difference between statistical fluid mechanics and data assimilation is the definition of 'truth' for the NWP model values that are required for a description of error statistics. Frehlich (2006) extended the definition of error statistics to the NWP data assimilation and forecasting problem by defining 'truth' as the convolution of the continuous atmospheric state variables by the spatial filter of the NWP model at each grid coordinate. This produces a description of total observation error in terms of the instrument error and the observation sampling error (related to the 'error of representativeness ' Lorenc, 1986;Daley, 1993;Cohn, 1997), which describes the error produced by the difference between the observation sampling pattern and the spatial average of the model that defines 'truth'.…”

“…The total observation-error statistics are state-dependent since they depend on the local turbulence statistics. Optimal data-assimilation algorithms have been produced to include the spatial variations of the total observation error based on the maximum-likelihood (ML) method and using modifications to the classical mean-square error techniques such as the Kalman filter (Frehlich, 2006). To include the spatial variations in the total observation error, two different definitions of error were proposed: one defined for each model coordinate r a and one based on the interpolation of the model coordinates to each observation coordinate r k .…”

“…conditioned by the local turbulence statistics and consistent with the definition of truth. The impact of imprecise knowledge of the true shape of the effective model filter will be determined for these optimal data-assimilation algorithms (Frehlich, 2006). Simple formulations will be used to introduce these concepts.…”