On the properties of ensemble forecast sensitivity to observations

Abstract

Evaluating impacts of observations on the skill of numerical weather prediction (NWP) is important. The Ensemble Forecast Sensitivity to Observation (EFSO) provides an efficient approach to diagnosing observation impacts, quantifying how much each observation improves or degrades a subsequent forecast with a given verification reference. This study investigates the sensitivity of EFSO impact estimates to the choice of the verification reference, using a global NWP system consisting of the Non‐hydrostatic Icosahedral Atmospheric Model (NICAM) and the Local Ensemble Transform Kalman Filter (LETKF). The EFSO evaluates observation impacts with the moist total energy norm and with recently proposed observation‐based verification metrics. The results show that each type of observation mainly contributes to the improvement of forecast departures of the observed variable maybe due to the limitation of localization in the EFSO. The EFSO overestimates the fraction of beneficial observations when verified with subsequent analyses, especially for shorter lead times such as 6 h. We may avoid this overestimation to some extent by verifying with observations, analyses from other data assimilation (DA) systems, or analyses of an independent run with the same DA system. In addition, this study demonstrates two important issues possibly leading to overestimating observation impacts. First, observation impacts would be overestimated if we apply relaxation‐to‐prior methods to the initial conditions of the ensemble forecasts in the EFSO; therefore, the ensemble forecasts in the EFSO should be independent of the ensemble forecasts in the DA cycle. Second, deterministic baseline forecasts of the EFSO, which represent the forecast without DA, should be initialized by the ensemble mean of the first guess at the analysis time, not by the previous analysis.

Evaluation of adjoint‐based observation impacts as a function of forecast length using an Observing System Simulation Experiment

“…(2018) and Kotsuki et al . (2019) have found that biases can have large effects when calculating FSO. The GMAO OSSE is not completely devoid of bias—there are some model biases that result from differences in model physics between the G5NR and the forecast model.…”

“…(2013) and Kotsuki et al . (2019) found higher fractions of beneficial observations, as much as 60–70%, for 6‐hr forecasts. The expectation is that, as the forecast length increases and the error growth reaches saturation, the fraction of beneficial observations will approach 0.50, as any individual observation may be considered to perturb the long‐term forecast field randomly.…”

“…They found that biases in the verification fields had strong effects on the estimated observation impacts. Kotsuki et al (2019) looked at verification methods with ensemble FSO for short forecasts of 6-12 hr, comparing self-analysis verification with verification with reanalysis and observations. In their study, using self-analysis verification resulted in overinflated fractions of beneficial observations, particularly at 6 hr.…”

Predictability of Record-Breaking Rainfall in Japan in July 2018: Ensemble Forecast Experiments with the Near-Real-Time Global Atmospheric Data Assimilation System NEXRA

“…The advection of the localization center (Ota et al 2013) is not applied. The EFSO impact estimates were not very sensitive to the choice of the RTPS parameter in the NICAM-LETKF (Kotsuki et al 2018b).…”

“…For this purpose, we use a data assimilation (DA) system comprising the Nonhydrostatic ICosahedral Atmospheric Model (NICAM; Satoh et al 2008Satoh et al , 2014 and the Local Ensemble Transform Kalman Filter (LETKF; Hunt et al 2007). The NICAM-LETKF system was developed by Terasaki et al (2015) and has been extended continuously to assimilate satellite radiance and precipitation data (Terasaki and Miyoshi 2017;Kotsuki et al 2017a), to estimate model parameters (Kotsuki et al 2018a), to improve its variance inflation method (Kotsuki et al 2017b), to diagnose assimilated observations (Kotsuki et al 2018b), and to accelerate its computation for high-performance computing systems (Yashiro et al 2016). Recently, a 100-member near-real-time NICAM-LETKF system started running on the Japan Aerospace eXploration Agency (JAXA)'s second-generation supercomputer system (JSS2).…”