A Sampling Approach for Four Dimensional Data Assimilation

Attia, Ahmed; Rao, Vishwas; Sandu, Adrian

doi:10.1007/978-3-319-25138-7_20

Cited by 9 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most direct approach is to write the new implementation completely in Python. This, however, may sacrifice efficiency, 10 or may not be feasible when existing code in other languages needs to be reused. One of the main characteristics of DATeS is the possibility of incorporating code written in low level languages.…”

Section: Extending Datesmentioning

confidence: 99%

“…We report the results over the second two-thirds of the experiments timespan, i.e. over the interval [10,30] to avoid spinup artifacts. This interval consisting of the last 200 assimilation cycles out of 300, will be referred to as the "testing timespan".…”

Section: Extending Datesmentioning

confidence: 99%

“…These Figure 16: Data assimilation results with DEnKF applied to Lorenz-96 system. The minimum average RMSE over the interval [10,30] is indicated by red triangles. The minimum average KL distance between the analysis rank histogram and a uniformly distributed rank histogram [10,30] is indicated by blue tripods.…”

Section: Extending Datesmentioning

confidence: 99%

See 2 more Smart Citations

DATeS: a highly extensible data assimilation testing suite v1.0

Attia

Sandu

2019

Geosci. Model Dev.

Self Cite

View full text Add to dashboard Cite

A flexible and highly-extensible data assimilation testing suite, named DATeS, is described in this paper. DATeS aims to offer a unified testing environment that allows researchers to compare different data assimilation methodologies and understand their performance in various settings. The core of DATeS is implemented in Python and takes advantage of its object-oriented capabilities. The main components of the package (the numerical models, the data assimilation algorithms, the linear algebra solvers, and the time discretization routines) are independent of each other, which offers great flexibility to configure data assimilation applications. DATeS can interface easily with large third-party numerical models written in Fortran or in C, and with a plethora of external solvers.

show abstract

Section: Extending Datesmentioning

confidence: 99%

Section: Extending Datesmentioning

confidence: 99%

See 1 more Smart Citation

DATeS: a highly extensible data assimilation testing suite v1.0

Attia

Sandu

2019

Geosci. Model Dev.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In general the posterior PDF (18) will not correspond to a Gaussian mixture due to the nonlinearity of the observation operator. This makes analytical solutions not possible.…”

Section: Cluster Sampling Filtersmentioning

confidence: 99%

Cluster Sampling Filters for Non-Gaussian Data Assimilation

2018

Self Cite

View full text Add to dashboard Cite

This paper presents a fully non-Gaussian filter for sequential data assimilation. The filter is named the "cluster sampling filter", and works by directly sampling the posterior distribution following a Markov Chain Monte-Carlo (MCMC) approach, while the prior distribution is approximated using a Gaussian Mixture Model (GMM). Specifically, a clustering step is introduced after the forecast phase of the filter, and the prior density function is estimated by fitting a GMM to the prior ensemble. Using the data likelihood function, the posterior density is then formulated as a mixture density, and is sampled following an MCMC approach. Four versions of the proposed filter, namely C MCMC, C HMC, MC-C HMC, and MC-C HMC are presented. C MCMC uses a Gaussian proposal density to sample the posterior, and C HMC is an extension to the Hamiltonian Monte-Carlo (HMC) sampling filter. MC-C MCMC and MC-C HMC are multi-chain versions of the cluster sampling filters C MCMC and C HMC respectively. The multi-chain versions are proposed to guarantee that samples are taken from the vicinities of all probability modes of the formulated posterior. The new methodologies are tested using a simple one-dimensional example, and a quasi-geostrophic (QG) model with double-gyre wind forcing and bi-harmonic friction. Numerical results demonstrate the usefulness of using GMMs to relax the Gaussian prior assumption especially in the HMC filtering paradigm.

show abstract

“…These values correspond (approximately) to a standard deviation of 5% of the timeaveraged values for u and v components, and 2% of the time-averaged values for h component. A flow dependent background error covariance matrix is constructed as described in [1,2]. The standard deviation of the background errors for the height component is 2% of the average magnitude of the height component in the reference initial condition.…”

Section: Admm Solution Formentioning

confidence: 99%

Robust Data Assimilation Using $L_1$ and Huber Norms

Rao

Sandu

Ng³

et al. 2017

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract. Data assimilation is the process to fuse information from priors, observations of nature, and numerical models, in order to obtain best estimates of the parameters or state of a physical system of interest. Presence of large errors in some observational data, e.g., data collected from a faulty instrument, negatively affect the quality of the overall assimilation results. This work develops a systematic framework for robust data assimilation. The new algorithms continue to produce good analyses in the presence of observation outliers. The approach is based on replacing the traditional L 2 norm formulation of data assimilation problems with formulations based on L 1 and Huber norms. Numerical experiments using the Lorenz-96 and the shallow water on the sphere models illustrate how the new algorithms outperform traditional data assimilation approaches in the presence of data outliers.1. Introduction. Dynamic data-driven application systems (DDDAS [3]) integrate computational simulations and physical measurements in symbiotic and dynamic feedback control systems. Within the DDDAS paradigm, data assimilation (DA) defines a class of inverse problems that fuses information from an imperfect computational model based on differential equations (which encapsulates our knowledge of the physical laws that govern the evolution of the real system), from noisy observations (sparse snapshots of reality), and from an uncertain prior (which encapsulates our current knowledge of reality). Data assimilation integrates these three sources of information and the associated uncertainties in a Bayesian framework to provide the posterior, i.e., the probability distribution conditioned on the uncertainties in the model and observations. Two approaches to data assimilation have gained widespread popularity: ensemblebased estimation and variational methods. The ensemble-based methods are rooted in statistical theory, whereas the variational approach is derived from optimal control theory. The variational approach formulates data assimilation as a nonlinear optimization problem constrained by a numerical model. The initial conditions (as well as boundary conditions, forcing, or model parameters) are adjusted to minimize the discrepancy between the model trajectory and a set of time-distributed observations. In real-time operational settings the data assimilation process is performed in cycles: observations within an assimilation window are used to obtain an optimal trajectory, which provides the initial condition for the next time window, and the process is repeated in the subsequent cycles.Large errors in some observations can adversely impact the overall solution to the data assimilation system, e.g., can lead to spurious features in the analysis [15]. Various factors contribute to uncertainties in observations. Faulty and malfunctioning

show abstract

A Sampling Approach for Four Dimensional Data Assimilation

Cited by 9 publications

References 11 publications

DATeS: a highly extensible data assimilation testing suite v1.0

DATeS: a highly extensible data assimilation testing suite v1.0

Cluster Sampling Filters for Non-Gaussian Data Assimilation

Robust Data Assimilation Using $L_1$ and Huber Norms

Contact Info

Product

Resources

About