Parallelization in the time dimension of four‐dimensional variational data assimilation

Fisher, Michael; Gürol, Selime

doi:10.1002/qj.2997

Cited by 45 publications

(84 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Armed with these concepts and notation, we may now consider the original saddle technique as discussed in Fisher and Gürol () (and also used in (Freitag and Green, )). It is outlined as Algorithm 3.1, where we define

r (δ λ, δ μ, δ x) = (\begin{matrix} boldD & bold0 & boldL \\ bold0 & boldR & boldH \\ L^{T} & H^{T} & bold0 \end{matrix}) (\begin{matrix} δ bold-italicλ \\ δ bold-italicμ \\ δ boldx \end{matrix}) - (\begin{matrix} boldb \\ boldd \\ bold0 \end{matrix}) .

…”

Section: The Original Saddle Methodsmentioning

confidence: 99%

“…A third version (the “saddle” formulation) is obtained by transforming the terms in Equation in a set of equality constraints and writing the Karush–Kuhn–Tucker conditions for the resulting constrained problem, leading to the large “saddle” linear system

((), \begin{matrix} array D & array 0 & array L \\ array 0 & array R & array H \\ array {boldL}^{normalT} & array {boldH}^{normalT} & array 0 \end{matrix}) ((), \begin{matrix} array δ λ \\ array δ μ \\ array δ x \end{matrix}) = ((), \begin{matrix} array b \\ array d \\ array 0 \end{matrix}),

where the control vector [ δ λ T , δ μ T , δ x T ] T is a (2 s + m )‐dimensional vector. For the sake of brevity, we do not cover the details of this latter derivation here (but see (Fisher and Gürol, ) and (Fisher et al. , )).…”

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

“…, )). We have chosen, for the present paper, to follow Fisher and Gürol () and to focus on the two simplest choices:

{\overset{M}{false}}_{i} = bold0 and {\overset{M}{false}}_{i} = boldI .

Once

\overset{L}{false}

is determined, it remains to decide on the complete form of the (left) preconditioner, depending on which of the formulations Equation is used. The remaining part of this section explains suitable preconditioners for different formulations.…”

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

“…P M being the inexact constraint preconditioner suggested in Bergamaschi et al. , (; ()) and used in Fisher and Gürol (), and P B and P T being the triangular and block‐diagonal ones inspired by Benzi and Wathen () (also Wathen ()). These inverses are given by

\begin{matrix} alignleft \\ align-1 {boldP}_{M}^{- 1} & align-2 = (\begin{matrix} bold0 & bold0 & {\overset{L}{false}}^{- 1} \\ bold0 & R^{- 1} & bold0 \\ {\overset{L}{false}}^{- T} & bold0 & - S^{- 1} \end{matrix}), \end{matrix}

\begin{matrix} alignleft \\ align-1 {boldP}_{B}^{- 1} & align-2 = (\begin{matrix} D^{- 1} & bold0 & bold0 \\ bold0 & R^{- 1} & bold0 \\ bold0 & bold0 & S^{- 1} \end{matrix}), \end{matrix}

\begin{matrix} alignleft \\ align-1 {boldP}_{T}^{- 1} & align-2 = (\begin{matrix} D^{- 1} & bold0 & - D^{- 1} \overset{L}{false} S^{- 1} \\ bold0 & R^{- 1} & - R^{- 1} boldH S^{- 1} \\ bold0 & bold0 & S^{- 1} \end{matrix}), \end{matrix}

and therefore both

…”

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

“…The “saddle formulation”, discussed in Fisher et al. , (; ()) Fisher and Gürol (), has recently attracted interest of practioners because it minimizes the number of calls to B −1 and

{boldQ}_{j}^{- 1}

and because of its very appealing potential for parallel computing, while still allowing a wide choice of preconditioners. However it is fair to say that numerical experience with this approach remains scarse so far, prompting a more detailed assessment.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Guaranteeing the convergence of the saddle formulation for weakly constrained 4D‐Var data assimilation

Gratton

Gürol

Simon

et al. 2018

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

This paper discusses convergence issues for the saddle variational formulation of the weakly constrained 4D‐Var method in data assimilation, a method whose main interests are its parallelizable nature and its limited use of the inverse of the correlation matrices. It is shown that the method, in its original form, may produce erratic results or diverge because of the inherent lack of monotonicity of the produced objective function values. Convergent, variationally coherent variants of the algorithm are then proposed which largely retain the desirable features of the original proposal, and the circumstances in which these variants may be preferable to other approaches is briefly discussed.

show abstract

r (δ λ, δ μ, δ x) = (\begin{matrix} boldD & bold0 & boldL \\ bold0 & boldR & boldH \\ L^{T} & H^{T} & bold0 \end{matrix}) (\begin{matrix} δ bold-italicλ \\ δ bold-italicμ \\ δ boldx \end{matrix}) - (\begin{matrix} boldb \\ boldd \\ bold0 \end{matrix}) .

…”

Section: The Original Saddle Methodsmentioning

confidence: 99%

((), \begin{matrix} array D & array 0 & array L \\ array 0 & array R & array H \\ array {boldL}^{normalT} & array {boldH}^{normalT} & array 0 \end{matrix}) ((), \begin{matrix} array δ λ \\ array δ μ \\ array δ x \end{matrix}) = ((), \begin{matrix} array b \\ array d \\ array 0 \end{matrix}),

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

“…, )). We have chosen, for the present paper, to follow Fisher and Gürol () and to focus on the two simplest choices:

{\overset{M}{false}}_{i} = bold0 and {\overset{M}{false}}_{i} = boldI .

Once

\overset{L}{false}

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

\begin{matrix} alignleft \\ align-1 {boldP}_{M}^{- 1} & align-2 = (\begin{matrix} bold0 & bold0 & {\overset{L}{false}}^{- 1} \\ bold0 & R^{- 1} & bold0 \\ {\overset{L}{false}}^{- T} & bold0 & - S^{- 1} \end{matrix}), \end{matrix}

\begin{matrix} alignleft \\ align-1 {boldP}_{B}^{- 1} & align-2 = (\begin{matrix} D^{- 1} & bold0 & bold0 \\ bold0 & R^{- 1} & bold0 \\ bold0 & bold0 & S^{- 1} \end{matrix}), \end{matrix}

\begin{matrix} alignleft \\ align-1 {boldP}_{T}^{- 1} & align-2 = (\begin{matrix} D^{- 1} & bold0 & - D^{- 1} \overset{L}{false} S^{- 1} \\ bold0 & R^{- 1} & - R^{- 1} boldH S^{- 1} \\ bold0 & bold0 & S^{- 1} \end{matrix}), \end{matrix}

and therefore both

…”

Section: Problem Formulations and Preconditioningmentioning

confidence: 99%

“…The “saddle formulation”, discussed in Fisher et al. , (; ()) Fisher and Gürol (), has recently attracted interest of practioners because it minimizes the number of calls to B −1 and

{boldQ}_{j}^{- 1}

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Guaranteeing the convergence of the saddle formulation for weakly constrained 4D‐Var data assimilation

Gratton

Gürol

Simon

et al. 2018

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

show abstract

The Advantages of Hybrid 4DEnVar in the Context of the Forecast Sensitivity to Initial Conditions

Song

Shin

et al. 2017

JGR Atmospheres

View full text Add to dashboard Cite

Hybrid four‐dimensional ensemble variational data assimilation (hybrid 4DEnVar) is a prospective successor to three‐dimensional variational data assimilation (3DVar) in operational weather prediction centers currently developing a new weather prediction model and those that do not operate adjoint models. In experiments using real observations, hybrid 4DEnVar improved Northern Hemisphere (NH; 20°N–90°N) 500 hPa geopotential height forecasts up to 5 days in a NH summer month compared to 3DVar, with statistical significance. This result is verified against ERA‐Interim through a Monte Carlo test. By a regression analysis, the sensitivity of 5 day forecast is associated with the quality of the initial condition. The increased analysis skill for midtropospheric midlatitude temperature and subtropical moisture has the most apparent effect on forecast skill in the NH including a typhoon prediction case. Through attributing the analysis improvements by hybrid 4DEnVar separately to the ensemble background error covariance (BEC), its four‐dimensional (4‐D) extension, and climatological BEC, it is revealed that the ensemble BEC contributes to the subtropical moisture analysis, whereas the 4‐D extension does to the midtropospheric midlatitude temperature. This result implies that hourly wind‐mass correlation in 6 h analysis window is required to extract the potential of hybrid 4DEnVar for the midlatitude temperature analysis to the maximum. However, the temporal ensemble correlation, in hourly time scale, between moisture and another variable is invalid so that it could not work for improving the hybrid 4DEnVar analysis.

show abstract

Online model error correction with neural networks in the incremental 4D-Var framework

Farchi

Chrust

Bocquet

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent studies have demonstrated that it is possible to combine machine learning with data assimilation to reconstruct the dynamics of a physical model partially and imperfectly observed. Data assimilation is used to estimate the system state from the observations, while machine learning computes a surrogate model of the dynamical system based on those estimated states. The surrogate model can be defined as an hybrid combination where a physical model based on prior knowledge is enhanced with a statistical model estimated by a neural network. The training of the neural network is typically done offline, once a large enough dataset of model state estimates is available. By contrast, with online approaches the surrogate model is improved each time a new system state estimate is computed. Online approaches naturally fit the sequential framework encountered in geosciences where new observations become available with time. In a recent methodology paper, we have developed a new weak-constraint 4D-Var formulation which can be used to train a neural network for online model error correction. In the present article, we develop a simplified version of that method, in the incremental 4D-Var framework adopted by most operational weather centres. The simplified method is implemented in the ECMWF Object-Oriented Prediction System, with the help of a newly developed Fortran neural network library, and tested with a two-layer two-dimensional quasi geostrophic model. The results confirm that online learning is effective and yields a more accurate model error correction than offline learning. Finally, the simplified method is compatible with future applications to state-of-the-art models such as the ECMWF Integrated Forecasting System.

show abstract

Parallelization in the time dimension of four‐dimensional variational data assimilation

Cited by 45 publications

References 40 publications

Guaranteeing the convergence of the saddle formulation for weakly constrained 4D‐Var data assimilation

Guaranteeing the convergence of the saddle formulation for weakly constrained 4D‐Var data assimilation

The Advantages of Hybrid 4DEnVar in the Context of the Forecast Sensitivity to Initial Conditions

Online model error correction with neural networks in the incremental 4D-Var framework

Contact Info

Product

Resources

About