Heterogeneous filtering of ensemble‐based background‐error variances

Abstract

Background-error variances estimated from a finite size ensemble of data assimilations are affected by sampling noise, which degrades the accuracy of the variance estimates. Previous work highlighted the close link between the spatial structures of background error and the associated sampling noise, and demonstrated the ability of local spatial averaging to remove this sampling noise.Existing filtering techniques commonly assume a homogeneous smoothing of the estimated variances. However, this assumption can be inadequate to represent error structures of varying scales, e.g. small-scale errors associated with localized severe weather events. To answer this problem, this article introduces and examines a heterogeneous filter based on the knowledge of the local spatial properties of the sampling noise. The filtering is realized with a diffusion process, and the diffusion coefficient is parametrized according to the local correlation length-scale of the sampling noise. This enables the diffusion coefficient to vary spatially in such a way as to encourage smoothing in regions where the background error is large scale in preference to regions where the error is small scale.A simulated 1D framework is considered to test the proposed approach. It is shown that the filtering using a spatially varying diffusion coefficient is able to preserve high-frequency variance structures, while this information tends to be smoothed with homogeneous filtering. The benefits of applying heterogeneous filtering are particularly pronounced with small ensemble sizes and in the vicinity of localized variance maxima.

The evolution of the ECMWF hybrid data assimilation system

“…The computational cost of running an ensemble of independent 4D‐Var cycles in the EDA makes it necessary to reduce the size of the ensemble and the spatial resolution (both in inner and outer loops) of its members so it can conveniently be run in the operational schedule. The sampling errors introduced in the estimation of B by the limited ensemble size, and methods to control them, have been extensively discussed for background‐error standard deviations (Berre et al , ; Raynaud et al , , , ; Bonavita et al , ; Pannekoucke et al , ) and for the background‐error correlation structures (Pannekoucke et al , , ; Varella et al , ). An aspect that has received less attention is the impact of the different, considerably smaller, resolution at which the EDA members are currently run with respect to the resolution of the target assimilation system whose error statistics they are used to simulate.…”

A wavelet‐based filtering of ensemble background‐error variances

“…For comparison purposes, Figure 11(a) presents the estimated variances after applying an optimized homogeneous filter. The signal is quite well represented by this homogeneous filter; however, as detailed in Raynaud and Pannekoucke (2012), the amplitude of the variance peak is indicates that the performance of the wavelet thresholding is comparable to that of an optimized heterogeneous diffusion-based filter. This similar performance is encouraging since the wavelet thresholding is easier to implement than the diffusion-based heterogeneous filter.…”

“…This similar performance is encouraging since the wavelet thresholding is easier to implement than the diffusion-based heterogeneous filter. The main advantage is that it does not require knowledge of local background-error length scales, as is the case for the parametrization of diffusion in Raynaud and Pannekoucke (2012).…”

“…This could be achieved by performing the filtering either in grid-point space or wavelet space. A first contribution to heterogeneous variance filtering in grid-point space has been proposed by Raynaud and Pannekoucke (2012) based on the integration of the heterogeneous diffusion equation. In the present paper, the application of nonlinear filtering in wavelet space is examined.…”

On the diffusion equation and its application to isotropic and anisotropic correlation modelling in variational assimilation

“…The filter in this example is very simple and the choice of filtering scale is somewhat ad hoc. More sophisticated (objective) filters could be expected to perform better as discussed by Raynaud et al (2009) and Berre and Desroziers (2010), and recently by Raynaud and Pannekoucke (2012) within the context of filters based on diffusion.…”