The ensemble Kalman filter (EnKF) is widely used for sequential data assimilation. It operates as a succession of forecast and analysis steps. In realistic large-scale applications, EnKFs are implemented with small ensembles and poorly known model error statistics. This limits their representativeness of the background error covariances and, thus, their performance. This work explores the efficiency of the one-step-ahead (OSA) smoothing formulation of the Bayesian filtering problem to enhance the data assimilation performance of EnKFs. Filtering with OSA smoothing introduces an updated step with future observations, conditioning the ensemble sampling with more information. This should provide an improved background ensemble in the analysis step, which may help to mitigate the suboptimal character of EnKF-based methods. Here, the authors demonstrate the efficiency of a stochastic EnKF with OSA smoothing for state estimation. They then introduce a deterministic-like EnKF-OSA based on the singular evolutive interpolated ensemble Kalman (SEIK) filter. The authors show that the proposed SEIK-OSA outperforms both SEIK, as it efficiently exploits the data twice, and the stochastic EnKF-OSA, as it avoids observational error undersampling. They present extensive assimilation results from numerical experiments conducted with the Lorenz-96 model to demonstrate SEIK-OSA’s capabilities.

Capsule Summary An integrated, high resolution, data-driven regional modeling system has been recently developed for the Red Sea region and is being used for research and various environmental applications.

The Kalman filter (KF) is derived under the assumption of time-independent (white) observation noise. Although this assumption can be reasonable in many ocean and atmospheric applications, the recent increase in sensor coverage, such as the launching of new constellations of satellites with global spatio-temporal coverage, will provide high density of oceanic and atmospheric observations which are expected to have time-dependent (coloured) error statistics. In this situation, the KF update has been shown to generally provide overconfident probability estimates, which may degrade the filter performance. Different KF-based schemes accounting for time-correlated observation noise were proposed for small systems by modelling the coloured noise as a first-order autoregressive model driven by white Gaussian noise. This work introduces new ensemble Kalman filters (EnKFs) which account for coloured observational noises for efficient data assimilation into large-scale oceanic and atmospheric applications.More specifically, we follow the standard and the one-step-ahead smoothing formulations of the Bayesian filtering problem with coloured observational noise, modelled as an autoregressive model, to derive two (deterministic) EnKFs. We demonstrate the relevance of the coloured observational noise-aware EnKFs and analyze their performances through extensive numerical experiments conducted with the Lorenz-96 model.

This work combines two auxiliary techniques, namely the one-step-ahead (OSA) smoothing and the hybrid formulation, to boost the forecasting skills of a storm surge ensemble Kalman filter (EnKF) forecasting system. Bayesian filtering with OSA-smoothing enhances the robustness of the ensemble background statistics by exploiting the data twice: first to constrain the sampling of the forecast ensemble with the future observation, and then to update the resulting ensemble. This is expected to improve the behavior of EnKF-like schemes during the strongly nonlinear surges periods, but requires integrating the ensemble with the forecast model twice, which could be computationally demanding. The hybrid flow-dependent/static formulation of the EnKF background error covariance is then considered to enable the implementation of the filter with a small flow-dependent ensemble size, and thus less model runs. These two methods are combined within an ensemble transform Kalman filter (ETKF). The resulting hybrid ETKF with OSA smoothing is tested, based on twin experiments, using a realistic setting of the Advanced Circulation (ADCIRC) model configured for storm surge forecasting in the Gulf of Mexico and assimilating pseudo-observations of sea surface levels from a network of buoys. The results of our numerical experiments suggest that the proposed filtering system significantly enhances ADCIRC forecasting skills compared to the standard ETKF without increasing the computational cost.

This study investigates the filtering problem with one‐way coupled (OWC) state‐space systems, for which the joint ensemble Kalman filter (EnKF) is the standard solution. In this approach, the states of the two coupled sub‐systems are jointly updated with all incoming observations. This enables transfer of the information across the sub‐systems, which should provide coupled‐state estimates in better agreement with the observations. The state estimates of the joint EnKF highly depend on the relevance of the joint ensembles' cross‐covariances between the sub‐systems' variables. In this work, we propose a new joint EnKF scheme based on the One‐Step‐Ahead (OSA) smoothing formulation of the filtering problem for efficient assimilation into OWC systems. The scheme introduces an extra smoothing step for both states' sub‐systems with the future observations, followed by an analysis step for each sub‐system state using only its own observation, all within a Bayesian‐consistent framework. The extra OSA smoothing step enables us to more efficiently exploit the observations, to enhance the representativeness of the EnKF covariances, and to mitigate for reported inconsistencies in the joint EnKF analysis step. We demonstrate the relevance of the proposed approach by presenting and analyzing results of various numerical experiments conducted with a OWC Lorenz‐96 model.

No abstract

This work addresses the problem of data assimilation in large‐dimensional systems with colored observation noise of unknown statistics, a scenario that will become more common in the near future with the deployment of denser observational networks of high spatio‐temporal coverage. Here, we are interested in the ensemble Kalman filter (EnKF) framework, which has been derived around a white observation noise assumption. Recently, colored observation‐noise aware EnKFs in which the noise was modeled as a first‐order autoregressive (AR) model were introduced. This work generalizes the above‐mentioned filters to learn the statistics of the AR model further online. We follow the state augmentation approach first to estimate the state and the AR model transfer matrix simultaneously, then the variational Bayesian approach to estimate the AR model noise covariance parameters further. We accordingly derive two filtering EnKF‐like algorithms, which estimate those statistics together with the system state. We demonstrate the effectiveness of the proposed colored observation‐noise aware filtering schemes and compare their performance based on several numerical experiments conducted with the Lorenz‐96 model.

A new Hybrid ensemble data assimilation system is implemented with a Massachusetts Institute of Technology general circulation model (MITgcm) of the Red Sea. The system is based on the Data Assimilation Research Testbed (DART) and combines a time-varying ensemble generated by the Ensemble Adjustment Kalman Filter (EAKF) with a pre-selected quasi-static (monthly varying) ensemble as used in an Ensemble Optimal Interpolation (EnOI) scheme. The goal is to develop an efficient system that enhances the state estimate and model forecasting skill in the Red Sea with reduced computational load compared to the EAKF. Observations of satellite sea-surface temperature (SST), altimeter sea-surface height (SSH), and in situ temperature and salinity profiles are assimilated to evaluate the new system. The performance of the Hybrid scheme (hereafter Hybrid-EAKF) is assessed with respect to the EnOI and the EAKF results. The comparisons are based on the daily averaged forecasts against satellite SST and SSH measurements and independent in situ temperature and salinity profiles. Hybrid-EAKF yields significant improvements in terms of ocean state estimates compared to both EnOI and EAKF, in particular mitigating for dynamical imbalances that affect EnOI. Hybrid-EAKF improves the estimation of SST and SSH root-mean-square differences by up to 20% compared to EAKF. High-resolution mesoscale eddy features, which dominate the Red Sea circulation, are further better represented in Hybrid-EAKF. Important reduction, by about 75%, in computational cost is also achieved with the Hybrid-EAKF system compared to the EAKF. These significant improvements were obtained with the Hybrid-EAKF after accounting for uncertainties in the atmospheric forcing and internal model physics in the time-varying ensemble.