The importance of appropriate verification metrics for the assessment of observation impact in a convection‐permitting modelling system

Abstract

Over the past 15 years, adjoint-based, ensemble-based and hybrid methods have been developed for estimating observation impact based on the forecast sensitivity to observation impact (FSOI). These methods are now commonly used in global modelling systems. However, little attention has been given to assessing observation impact in regional convection-permitting modelling systems. This study presents the first evaluation of ensemble-based estimates of observation impact over an extended period of six weeks in such a convection-permitting modelling system, namely the regional ensemble system of Deutscher Wetterdienst. Another aspect that has received little attention is the choice of the forecast-error verification metric. Nearly all previous studies used the difference between the forecast and a subsequent analysis of the same modelling system expressed in terms of energy (total energy norm). While such a self-verification generally needs to be treated with caution, it appears unsuitable for convection-permitting regional forecasts. Firstly, total energy does not really reflect parameters that forecast users are interested in, and important forecast quantities such as surface wind gusts and precipitation are not even part of the analysis. Secondly, systematic analysis and forecast errors are non-negligible in the presence of convection, especially for important variables that are related to convection. To overcome this issue, we introduce the use of independent radar observations for the verification of observation impact and compare results for a variety of different observation-based metrics for a six-week high-impact weather period in summer 2016. This revealed a particular sensitivity of the estimated impact to model as well as observation biases and sensitivity studies indicated that even small biases can have an influence on the estimated impact. Additionally, we demonstrate that FSOI can be used to identify biases through comparison of results for different metrics.

Assimilating visible satellite images for convective‐scale numerical weather prediction: A case‐study

“…The data assimilation experiments were conducted for two days, namely 29 May and 5 June 2016, from a highly convective period that has been studied extensively (e.g., Necker et al . 2018; 2020; Keil et al . 2019; Bachmann et al .…”

Assimilation of SEVIRI Water Vapor Channels With an Ensemble Kalman Filter on the Convective Scale

“…The first is a summer period in the year 2016 from May 26, 2016 to June 30, 2016. It exhibited extreme weather situations ranging from highly convective situations with strong precipitation events including flash floods and thunder storms primarily from end of May for 3 weeks to hot and dry weather primarily toward end of June 2016 (Piper et al, 2016;Necker et al, 2018). The cloud mask analysis in Figure 7 gives further details on the occurrence of cloud types during the summer period.…”

Assimilating Visible and Infrared Radiances in Idealized Simulations of Deep Convection

“…For data assimilation, we use the local ensemble transform Kalman filter (LETKF; Hunt et al 2007) implemented in the kilometer-scale ensemble data assimilation system KENDA for the operational regional model, Consortium for Small-Scale Modeling (COSMO) of Deutscher Wetterdienst (Schraff et al 2016). The COSMO-KENDA system is operational at Deutscher Wetterdienst and has been used for a number of assimilation studies (Schomburg et al 2015;Necker et al 2018;Weissmann 2014, 2016;Hutt et al 2020). To calculate synthetic infrared satellite observations from the model state, we simulate the cloud-affected infrared radiances with the radiative transfer code RTTOV (Saunders et al 1999;Matricardi and Saunders 1999).…”

“…For synthetic observations in the visible channel, we use the method Method for Fast Satellite Image Simulation (MFASIS) recently put forward by Scheck et al (2016Scheck et al ( , 2018, which is by now also included in RTTOV. Compared to the assimilation of conventional observations (Schraff et al 2016;Necker et al 2018), a larger number (.6000) of all-sky radiance measurements can be assimilated every hour in a model domain covering (e.g., central Europe).…”