Enhanced parallelization of the incremental 4D‐Var data assimilation algorithm using the Randomized Incremental Optimal Technique

Abstract

Incremental 4D-Var is a data assimilation algorithm used routinely at operational numerical weather prediction (NWP) centres worldwide. The algorithm solves a series of quadratic minimization problems (inner-loops) obtained from linear approximations of the forward model around nonlinear trajectories (outer-loops). Since most of the computational burden is associated with the inner-loops, many studies have focused on developing computationally efficient algorithms to solve the least-square quadratic minimization problem, in particular through time parallelization. This paper presents the first implementation and testing of a recently proposed method for parallelizing incremental 4D-Var, the Randomized Incremental Optimal Technique (RIOT), which replaces the traditional sequential conjugate gradient (CG) iterations in the inner-loop of the minimization with fully parallel randomized singular value decomposition (RSVD) of the preconditioned Hessian of the cost function. RIOT is tested using the standard Lorenz-96 model (L-96) as well as two realistic high-dimensional atmospheric source inversion problems based on aircraft observations of black carbon concentrations. A new outer-loop preconditioning technique tailored to RSVD was introduced to improve convergence stability and performance. Results obtained with the L-96 system show that the performance improvement from RIOT compared to standard CG algorithms increases significantly with nonlinearities. Overall, in the realistic black carbon source inversion experiments, RIOT reduces the wall-clock time of the 4D-Var minimization by a factor of 2 to 3, at the cost of a factor of 4 to 10 increase in energy cost due to the large number of parallel cores used. Furthermore, RIOT enables reduction of the wall-clock time computation of the analysis-error covariance matrix by a factor of 40 compared to a standard iterative Lanczos approach. Finally, as evidenced in this study, implementation of RIOT in an operational NWP system will require a better understanding of its convergence properties as a function of the Hessian characteristics and, in particular, the degree of freedom for signal of the inverse problem.

Randomised preconditioning for the forcing formulation of weak‐constraint 4D‐Var

“…A different approach has been explored by Bousserez and Henze (2018) and Bousserez et al . (2020), who presented and tested a randomised solution algorithm called the Randomized Incremental Optimal Technique (RIOT) in data assimilation. RIOT is designed to be used instead of PCG and employs a randomised eigenvalue decomposition of the Hessian (using a different method from the ones presented in this article) to construct directly the solution in Equation (), which approximates the solution given by PCG.…”

“…In this work we apply randomised methods to generate a preconditioner, which is then used to accelerate the solution of the exact inner loop problem (11) with PCG method (as discussed in Section 4). A different approach has been explored by Bousserez and Henze (2018) and Bousserez et al (2020), who presented and tested a randomised solution algorithm called the Randomized Incremental Optimal Technique (RIOT) in data assimilation. RIOT is designed to be used instead of PCG and employs a randomised eigenvalue decomposition of the Hessian (using a different method than the ones presented in this paper) to directly construct the solution x in (11), which approximates the solution given by PCG.…”

On time‐parallel preconditioning for the state formulation of incremental weak constraint 4D‐Var

“…Randomised methods for low‐rank matrix approximations have attracted a lot of interest in recent years because they require matrix products with blocks of vectors that can be easily parallelised, and it has been shown that good approximations for matrices with rapidly decaying singular values can be obtained with high probability (e.g., Halko et al ., 2011, Martinsson and Tropp 2020). These methods have been explored in data assimilation when designing solvers for strong constraint 4D‐Var (Bousserez et al ., 2020) and preconditioning for the forcing formulation of the incremental weak constraint 4D‐Var (Daužickaitė et al ., 2021).…”

“…In the light of the growing popularity of randomised methods and examples of their use in data assimilation (Bousserez et al ., 2020, Daužickaitė et al ., 2021), we propose using a randomised singular value decomposition (RSVD; Halko et al ., 2011) to approximate the tangent linear model. RSVD is a block method that is easy to parallelise in the sense that it requires calculating matrix products with blocks of vectors.…”

Randomised preconditioning for the forcing formulation of weak constraint 4D-Var

“…In the light of the growing popularity of randomised methods and examples of their use in data assimilation (Bousserez et al [2020], Daužickaitė et al [2021]), we propose using a randomised singular value decomposition (RSVD) (Halko et al [2011]) to approximate the tangent linear model. RSVD is a block method that is easy to parallelise in the sense that it requires calculating matrix products with blocks of vectors.…”