An Iterative Ensemble Kalman Smoother in Presence of Additive Model Error

Fillion, Anthony; Bocquet, Marc; Gratton, Serge; Gürol, Selime; Sakov, Pavel

doi:10.1137/19m1244147

Cited by 13 publications

(21 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also remark that the decoupling of w z and w q when the observation operator is linear, which was put forward and explained in Section 3 of [53] and Appendix B of [27], also happens for the IEnKF-ML.…”

Section: 24mentioning

confidence: 71%

“…Note that depending on the definition of the model noise statistics Ξ 1 , stochastic perturbations are also possible for the propagation of the surrogate model parameters, even though the deterministic parameter evolution model is persistence. Thanks to the augmented state trick, (27) has exactly the same form as that of the IEnKF-Q, so that we closely follow the derivation of its analysis step. We introduce the state model perturbation matrix as X q 1 ∈ R Nz×Nq and it is defined through…”

Section: 24mentioning

confidence: 99%

“…the parameter propagation is persistence only, without additional stochastic perturbations. In that case, (27) reads…”

Section: 24mentioning

confidence: 99%

See 2 more Smart Citations

Online learning of both state and dynamics using ensemble Kalman filters

Bocquet¹,

Farchi²,

Malartic³

2021

FoDS

Self Cite

View full text Add to dashboard Cite

<p style='text-indent:20px;'>The reconstruction of the dynamics of an observed physical system as a surrogate model has been brought to the fore by recent advances in machine learning. To deal with partial and noisy observations in that endeavor, machine learning representations of the surrogate model can be used within a Bayesian data assimilation framework. However, these approaches require to consider long time series of observational data, meant to be assimilated all together. This paper investigates the possibility to learn both the dynamics and the state online, i.e. to update their estimates at any time, in particular when new observations are acquired. The estimation is based on the ensemble Kalman filter (EnKF) family of algorithms using a rather simple representation for the surrogate model and state augmentation. We consider the implication of learning dynamics online through (ⅰ) a global EnKF, (ⅰ) a local EnKF and (ⅲ) an iterative EnKF and we discuss in each case issues and algorithmic solutions. We then demonstrate numerically the efficiency and assess the accuracy of these methods using one-dimensional, one-scale and two-scale chaotic Lorenz models.</p>

show abstract

Section: 24mentioning

confidence: 71%

Section: 24mentioning

confidence: 99%

See 1 more Smart Citation

Online learning of both state and dynamics using ensemble Kalman filters

Bocquet¹,

Farchi²,

Malartic³

2021

FoDS

Self Cite

View full text Add to dashboard Cite

show abstract

“…4.4. Likewise, it is possible that some of the issues faced by the IEnKS in integrating localization / hybridization (Bocquet, 2016) ( Sakov et al, 2018;Fillion et al, 2020), EnKS expectation maximization schemes (Pulido et al, 2018) or the family of OSA smoothers (Gharamti et al, 2015;Ait-El-Fquih et al, 2016;Raboudi et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

A fast, single-iteration ensemble Kalman smoother for sequential data assimilation

Grudzien

Bocquet

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. Ensemble-variational methods form the basis of the state-of-the-art for nonlinear, scalable data assimilation, yet current designs may not be cost-effective for reducing prediction error in online, short-range forecast systems. We propose a novel, outer-loop optimization of the ensemble-variational formalism for applications in which forecast error dynamics are weakly nonlinear, such as synoptic meteorology. In order to rigorously derive our method and demonstrate its novelty, we review ensemble smoothers that appear throughout the literature in a unified Bayesian maximum-a-posteriori narrative, updating and simplifying some results. After mathematically deriving our technique, we systematically develop and inter-compare all studied schemes in the open-source Julia package DataAssimilationBenchmarks.jl, with pseudo-code provided for these methods. This high-performance numerical framework, supporting our mathematical results, produces extensive benchmarks that demonstrate the significant performance advantages of our proposed technique. In particular, our single-iteration ensemble Kalman smoother is shown both to improve prediction / posterior accuracy and to simultaneously reduce the leading order cost of iterative, sequential smoothers in a variety of relevant test cases for operational short-range forecasts. This long work is thus intended to present our novel single-iteration ensemble Kalman smoother, and to provide a theoretical and computational framework for the study of sequential, ensemble-variational Kalman filters and smoothers generally.

show abstract

“…Thus, ensembles enable to approximate the forecast covariance matrix thanks to a reduced set of sample vectors. In this section, we describe a tailored version of the widely used ensemble algorithm ETKF (Bishop et al ., 2001), namely ETKF‐Q (Fillion et al ., 2020). What we call the ETKF‐Q method precisely denotes the IEnKS‐Q algorithm of (Fillion et al ., 2020, Algorithm 4.1) with parameters (

L = 0, K = 0, S = 1, G = 0

, one Gauss–Newton loop, transform version).…”

Section: Data Assimilation Within a Latent Spacementioning

confidence: 99%

Latent space data assimilation by using deep learning

Peyron

Fillion

Gürol

et al. 2021

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

Performing data assimilation (DA) at low cost is of prime concern in Earth system modeling, particularly in the era of Big Data, where huge quantities of observations are available. Capitalizing on the ability of neural network techniques to approximate the solution of partial differential equations (PDEs), we incorporate deep learning (DL) methods into a DA framework. More precisely, we exploit the latent structure provided by autoencoders (AEs) to design an ensemble transform Kalman filter with model error (ETKF‐Q) in the latent space. Model dynamics are also propagated within the latent space via a surrogate neural network. This novel ETKF‐Q‐Latent (ETKF‐Q‐L) algorithm is tested on a tailored instructional version of Lorenz 96 equations, named the augmented Lorenz 96 system, which possesses a latent structure that accurately represents the observed dynamics. Numerical experiments based on this particular system evidence that the ETKF‐Q‐L approach both reduces the computational cost and provides better accuracy than state‐of‐the‐art algorithms such as the ETKF‐Q.

show abstract

An Iterative Ensemble Kalman Smoother in Presence of Additive Model Error

Cited by 13 publications

References 14 publications

Online learning of both state and dynamics using ensemble Kalman filters

Online learning of both state and dynamics using ensemble Kalman filters

A fast, single-iteration ensemble Kalman smoother for sequential data assimilation

Latent space data assimilation by using deep learning

Contact Info

Product

Resources

About