Assimilation of semi‐qualitative observations with a stochastic ensemble Kalman filter

Abstract

The ensemble Kalman filter assumes observations to be Gaussian random variables with a pre-specified mean and variance. In practice, observations may also have detection limits, for instance when a gauge has a minimum or maximum value. In such cases, most data assimilation schemes discard out-of-range values, treating them as "not a number," with the loss of possibly useful qualitative information. The current work focuses on the development of a data assimilation scheme that tackles observations with a detection limit. We present the Ensemble Kalman Filter Semi-Qualitative (EnKF-SQ) and test its performance against the Partial Deterministic Ensemble Kalman Filter (PDEnKF) of Borup et al. Both are designed to assimilate out-of-range observations explicitly: the out-of-range values are qualitative by nature (inequalities), but one can postulate a probability distribution for them and then update the ensemble members accordingly. The EnKF-SQ is tested within the framework of twin experiments, using both linear and nonlinear toy models. Different sensitivity experiments are conducted to assess the influence of the ensemble size, observation detection limit and number of observations on the performance of the filter. Our numerical results show that assimilating qualitative observations using the proposed scheme improves the overall forecast mean, making it viable for testing on more realistic applications such as sea-ice models.

Assimilation of semi-qualitative sea ice thickness data with the EnKF-SQ: a twin experiment

“…The EnKF-SQ (Shah et al, 2018) uses an ensemble of model states to estimate the error statistics closely following the stochastic EnKF algorithm (Burgers et al, 1998;Evensen, 2004). The stochastic EnKF is a two-step filtering method alternating forecast and analysis steps.…”

“…Repeating this process for all forecast members yields the analysis ensemble. For a detailed description of the EnKF-SQ the reader is referred to Section 2 of Shah et al (2018).…”

“…Since all assimilation runs use perturbed winds, this effect is likely to impact the EnKF-IG and EnKF-SQ more than the EnKF-ALL and EnKF-CLIM. In addition, it is important to mention that there are fewer grid points (not shown here) in the bins for thicker ice compared to thin ice, which may also affect the estimation of the bias for these bins, making As discussed in Shah et al (2018), the two-piece Gaussian observation likelihood may influence the shape of the posterior distribution, making it skewed and thus less Gaussian. In order to examine this, we evaluate and plot the conditional skewness of the posterior estimates of SIT only at the last assimilation step in Figure 10.…”

“…With a new method intended for semi-qualitative data as the EnKF-SQ, the question arise whether this bias can be mitigated or not? The comparison of the EnKF-SQ to the perfect Bayesian solution (Shah et al, 2018) shows that the EnKF-SQ analysis does not coincide with the Bayesian posterior and bears inherent biases: in the case of hard data, the Bayesian and EnKF-SQ posteriors are nearly the same. However, for out-of-range observations and mode of a prior within the observable range, only the maximum likelihood of the EnKF-SQ analysis is preserved but its distribution is flatter than the Bayesian solution with a thicker tail in the unobservable range, so the expectation is too high.…”

“…For instance, the ice could be thicker than a known threshold. Studies from Shah et al (2018) and Borup et al (2015) have shown that assimilating soft data with linear and non-linear toy models using ensemble-based DA methods have the potential to improve the accuracy of the forecast. Therefore, not considering soft data in the assimilation procedure is a potential loss of meaningful information.…”