Ensemble-based approximation of observation impact using an observation-based verification metric

Self Cite

Over the past 15 years, adjoint-based, ensemble-based and hybrid methods have been developed for estimating observation impact based on the forecast sensitivity to observation impact (FSOI). These methods are now commonly used in global modelling systems. However, little attention has been given to assessing observation impact in regional convection-permitting modelling systems. This study presents the first evaluation of ensemble-based estimates of observation impact over an extended period of six weeks in such a convection-permitting modelling system, namely the regional ensemble system of Deutscher Wetterdienst. Another aspect that has received little attention is the choice of the forecast-error verification metric. Nearly all previous studies used the difference between the forecast and a subsequent analysis of the same modelling system expressed in terms of energy (total energy norm). While such a self-verification generally needs to be treated with caution, it appears unsuitable for convection-permitting regional forecasts. Firstly, total energy does not really reflect parameters that forecast users are interested in, and important forecast quantities such as surface wind gusts and precipitation are not even part of the analysis. Secondly, systematic analysis and forecast errors are non-negligible in the presence of convection, especially for important variables that are related to convection. To overcome this issue, we introduce the use of independent radar observations for the verification of observation impact and compare results for a variety of different observation-based metrics for a six-week high-impact weather period in summer 2016. This revealed a particular sensitivity of the estimated impact to model as well as observation biases and sensitivity studies indicated that even small biases can have an influence on the estimated impact. Additionally, we demonstrate that FSOI can be used to identify biases through comparison of results for different metrics.

“…Following Sommer and Weissmann (), the observation impact J of any subset of observations d ′ can be computed as:

\begin{matrix} alignleft \\ align-1 J ({boldd}^{'}) & align-2 \approx \frac{2}{N_{e} - 1} {bolde}_{f}^{boldd} \cdot {boldY}_{f}^{boldd} {({boldY}_{a}^{boldd})}^{T} {boldR}^{- 1} {boldd}^{'}, \end{matrix}

Section: Methods and Experimental Set‐upmentioning

confidence: 99%

Section: Methods and Experimental Set‐upmentioning

confidence: 99%

Section: Methods and Experimental Set‐upmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The importance of appropriate verification metrics for the assessment of observation impact in a convection‐permitting modelling system

Necker¹,

Weißmann²,

Sommer³

2018

Self Cite

“…observations minus forecasts) for the verification metrics. Sommer and Weissmann (, hereafter SW16) and Necker et al (, hereafter N18) tested the observation‐based metrics for the Deutscher Wetterdienst (DWD) operational regional ensemble DA system. Similarly, Cardinali (, hereafter C18) introduced an observation‐based metric for the adjoint FSO of the European Centre for Medium‐range Weather Forecasting (ECMWF)'s global system.…”

Section: Introductionmentioning

confidence: 99%

On the properties of ensemble forecast sensitivity to observations

Kotsuki

Kurosawa

Miyoshi

2019

Evaluating impacts of observations on the skill of numerical weather prediction (NWP) is important. The Ensemble Forecast Sensitivity to Observation (EFSO) provides an efficient approach to diagnosing observation impacts, quantifying how much each observation improves or degrades a subsequent forecast with a given verification reference. This study investigates the sensitivity of EFSO impact estimates to the choice of the verification reference, using a global NWP system consisting of the Non‐hydrostatic Icosahedral Atmospheric Model (NICAM) and the Local Ensemble Transform Kalman Filter (LETKF). The EFSO evaluates observation impacts with the moist total energy norm and with recently proposed observation‐based verification metrics. The results show that each type of observation mainly contributes to the improvement of forecast departures of the observed variable maybe due to the limitation of localization in the EFSO. The EFSO overestimates the fraction of beneficial observations when verified with subsequent analyses, especially for shorter lead times such as 6 h. We may avoid this overestimation to some extent by verifying with observations, analyses from other data assimilation (DA) systems, or analyses of an independent run with the same DA system. In addition, this study demonstrates two important issues possibly leading to overestimating observation impacts. First, observation impacts would be overestimated if we apply relaxation‐to‐prior methods to the initial conditions of the ensemble forecasts in the EFSO; therefore, the ensemble forecasts in the EFSO should be independent of the ensemble forecasts in the DA cycle. Second, deterministic baseline forecasts of the EFSO, which represent the forecast without DA, should be initialized by the ensemble mean of the first guess at the analysis time, not by the previous analysis.

Survey of data assimilation methods for convective‐scale numerical weather prediction at operational centres

Gustafsson

Janjić

Schraff

et al. 2018

Self Cite

230

224

Data assimilation (DA) methods for convective‐scale numerical weather prediction at operational centres are surveyed. The operational methods include variational methods (3D‐Var and 4D‐Var), ensemble methods (LETKF) and hybrids between variational and ensemble methods (3DEnVar and 4DEnVar). At several operational centres, other assimilation algorithms, like latent heat nudging, are additionally applied to improve the model initial state, with emphasis on convective scales. It is demonstrated that the quality of forecasts based on initial data from convective‐scale DA is significantly better than the quality of forecasts from simple downscaling of larger‐scale initial data. However, the duration of positive impact depends on the weather situation, the size of the computational domain and the data that are assimilated. Furthermore it is shown that more advanced methods applied at convective scales provide improvements over simpler methods. This motivates continued research and development in convective‐scale DA. Challenges in research and development for improvements of convective‐scale DA are also reviewed and discussed. The difficulty of handling the wide range of spatial and temporal scales makes development of multi‐scale assimilation methods and space–time covariance localization techniques important. Improved utilization of observations is also important. In order to extract more information from existing observing systems of convective‐scale phenomena (e.g. weather radar data and satellite image data), it is necessary to provide improved statistical descriptions of the observation errors associated with these observations.