Tuning Covariance Localization Using Machine Learning

Moosavi, Azam S. Zavar; Attia, Ahmed; Sandu, Adrian

doi:10.1007/978-3-030-22747-0_16

Cited by 10 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While covariance localization has been useful for ameliorating spurious correlations over long distances, they require exhaustive tuning of the localization radius for determining at which point the spurious correlations should be cut off [15,16]. More recently, to avoid the empirical tuning of the localization parameters, several schemes of adaptive localization have been proposed and validated for ensemble data assimilation depending on their specific implementations [17][18][19][20][21][22], e.g., using an ensemble of ensembles to mitigate spurious correlations such as sampling error correction and reducing correlation sampling for adaptive localization [23][24][25][26]; assuming the localization function can be taken as a power of the background error correlation function [27][28][29]; proposing the theory of optimal linear filtering for adaptive localization [30]; proposing a correlation-based adaptive localization method [31][32][33]; and considering machine learning algorithms to capture the adaptive localization parameters [34][35][36]. The adaptive localization technique has been developed in detail, but some of these methods make strong assumptions or do not always work in ensemble data assimilation (EDA)-like frameworks.…”

Section: Introductionmentioning

confidence: 99%

An Investigation of Adaptive Radius for the Covariance Localization in Ensemble Data Assimilation

Xing

Liu

Zhang

et al. 2021

JMSE

View full text Add to dashboard Cite

The covariance matrix estimated from the ensemble data assimilation always suffers from filter collapse because of the spurious correlations induced by the finite ensemble size. The localization technique is applied to ameliorate this issue, which has been suggested to be effective. In this paper, an adaptive scheme for Schur product covariance localization is proposed, which is easy and efficient to implement in the ensemble data assimilation frameworks. A Gaussian-shaped taper function is selected as the localization taper function for the Schur product in the adaptive localization scheme, and the localization radius is obtained adaptively through a certain criterion of correlations with the background ensembles. An idealized Lorenz96 model with an ensemble Kalman filter is firstly examined, showing that the adaptive localization scheme helps to significantly reduce the spurious correlations in the small ensemble with low computational cost and provides accurate covariances that are similar to those derived from a much larger ensemble. The investigations of adaptive localization radius reveal that the optimal radius is model-parameter-dependent, vertical-level-dependent and nearly flow-dependent with weather scenarios in a realistic model; for example, the radius of model parameter zonal wind is generally larger than that of temperature. The adaptivity of the localization scheme is also illustrated in the ensemble framework and shows that the adaptive scheme has a positive effect on the assimilated analysis as the well-tuned localization.

show abstract

Section: Introductionmentioning

confidence: 99%

An Investigation of Adaptive Radius for the Covariance Localization in Ensemble Data Assimilation

Xing

Liu

Zhang

et al. 2021

JMSE

View full text Add to dashboard Cite

show abstract

“…A data-driven approach is thus similar in spirit to the grid-search because a set of observations is required and processed during training. Similarly, one can also use machine learning to find appropriate localizations Moosavi et al (2019). Empirical localization functions (Lei et al, 2015), can also be use and are constructed, empirically, from a training set of obserations.…”

Section: Localization and Inflationmentioning

confidence: 99%

Using global Bayesian optimization in ensemble data assimilation: parameter estimation, tuning localization and inflation, or all of the above

Lunderman

Morzfeld

Posselt

2021

Tellus A: Dynamic Meteorology and Oceanography

View full text Add to dashboard Cite

Global Bayesian optimization (GBO) is a derivative-free optimization method that is used widely in the techindustry to optimize objective functions that are expensive to evaluate, numerically or otherwise. We discuss the use of GBO in ensemble data assimilation (DA), where the goal is to update the state of a numerical model in view of noisy observations. Specifically, we consider three tasks: (i) the estimation of model parameters; (ii) the tuning of localization and inflation in ensemble DA; (iii) doing both, i.e. estimating model parameters while simultaneously tuning the localization and inflation of the ensemble DA. For all three tasks, the GBO works 'offline', i.e. a set of 'training' observations are used within GBO to determine appropriate model or localization/inflation parameters, which are subsequently deployed within an ensemble DA system. Because of the offline nature of the technique, GBO can easily be combined with existing DA systems and it can effectively decouple (nearly) linear/Gaussian aspects of a problem from highly nonlinear/ non-Gaussian ones. We illustrate the use of GBO in simple numerical experiments with the classical Lorenz problems. Our main goals are to introduce GBO in the context of ensemble DA and to spark an interest in GBO and its uses for streamlining important tasks in ensemble DA.

show abstract

“…The ensemble size, i.e., the number of physics-based model runs, is typically the main factor that limits the efficiency of EnKF. For increasing the quality of the results when ensembles are small, heuristics correction methods such as covariance shrinkage [10][11][12] and localization [13][14][15] have been developed. As some form of heuristic correction is required for operation implementations of the ensemble Kalman filter, reducing the need for such heuristic corrections in operational implementations is an important and active area of research.…”

Section: Introductionmentioning

confidence: 99%

Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Theory-Guided Autoencoders

Popov¹,

Sandu²

2022

Front. Appl. Math. Stat.

View full text Add to dashboard Cite

Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends the MFEnKF into to make use of a broader class of surrogate model such as those based on machine learning methods such as autoencoders non-linear couplings in between the model hierarchies. We identify the right-invertibility property for autoencoders as being a key predictor of success in the forecasting power of autoencoder-based reduced order models. We propose a methodology that allows us to construct reduced order surrogate models that are more accurate than the ones obtained via conventional linear methods. Numerical experiments with the canonical Lorenz'96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity ensemble Kalman filtering. We additionality show a large-scale proof-of-concept result with the quasi-geostrophic equations, showing the competitiveness of the method with a traditional reduced order model-based MFEnKF.

show abstract

Tuning Covariance Localization Using Machine Learning

Cited by 10 publications

References 29 publications

An Investigation of Adaptive Radius for the Covariance Localization in Ensemble Data Assimilation

An Investigation of Adaptive Radius for the Covariance Localization in Ensemble Data Assimilation

Using global Bayesian optimization in ensemble data assimilation: parameter estimation, tuning localization and inflation, or all of the above

Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Theory-Guided Autoencoders

Contact Info

Product

Resources

About