Comparison of Gaussian, logarithmic transform and mixed Gaussian–log‐normal distribution based 1DVAR microwave temperature–water‐vapour mixing ratio retrievals

Observations and predictions of near-zero non-negative variables such as aerosol, water vapour, cloud, precipitation and plankton concentrations have uncertainty distributions that are skewed and better approximated by gamma and inverse-gamma probability distribution functions (pdfs) than Gaussian pdfs. Current Ensemble Kalman Filters (EnKFs) yield suboptimal state estimates for these variables. Here, we introduce a variation on the EnKF that accurately solves Bayes' theorem in univariate cases where the prior forecasts and error-prone observations given truth come in (gamma, inverse-gamma) or (inversegamma, gamma) or (Gaussian, Gaussian) distribution pairs. Since its multivariate extension is similar to an EnKF, we refer to it as the GIGG-EnKF or GIGG where GIGG stands for Gamma, Inverse-Gamma and Gaussian. The GIGG-EnKF enables near-zero semi-positivedefinite variables with highly skewed uncertainty distributions to be assimilated without the need for observation bias inducing log-normal or Gaussian anamorphosis nonlinear transformations. In the special case that all observations are treated as Gaussian, the GIGG-EnKF gives identical results to the original EnKF. A multi-grid-point and multi-variable idealized system was used to compare and contrast the data assimilation performance of the GIGG with that of both the perturbed observation and deterministic forms of the EnKF. This test system featured variables and observation types whose uncertainty distributions approximate Gaussian, gamma and inverse-gamma distributions. The normalized analysis error variance of the GIGG ensemble mean was found to be significantly smaller than that of the EnKFs. The higher moments of the analysed ensemble distributions were tested by subjecting the ensemble members to nonlinear 'forecast' mappings. The normalized mean square error of the mean of the corresponding GIGG forecast ensemble was found to be less than a 3rd of that obtained from either form of the original EnKF.

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The GIGG‐EnKF: ensemble Kalman filtering for highly skewed non‐negative uncertainty distributions

Bishop¹

2016

“…A number of systems do use (or have previously used) log humidity as a control parameter, e.g. EC's global and regional systems (Gauthier et al ., ; Fillion et al ., ), JMA's global system (JMA, ), MM5 (Xiao et al ., ), WRF (Xiao and Sun, ), some 1D‐Var systems (Poli et al ., ; Deblonde and English, ; Kliewer et al ., ), radar‐based liquid water retrieval methods (e.g. Fielding et al ., , used with an EnKF), and ocean ecosystem models (e.g.…”

Section: Non‐gaussianitymentioning

confidence: 99%

Techniques and challenges in the assimilation of atmospheric water observations for numerical weather prediction towards convective scales

Bannister

Chipilski

Martínez‐Alvarado

2019

While contemporary numerical weather prediction models represent the large‐scale structure of moist atmospheric processes reasonably well, they often struggle to maintain accurate forecasts of small‐scale features such as convective rainfall. Even though high‐resolution models resolve more of the flow, and are therefore arguably more accurate, moist convective flow becomes increasingly nonlinear and dynamically unstable. Importantly, the models' initial conditions are typically sub‐optimal, leaving scope to improve the accuracy of forecasts with improved data assimilation. To address issues regarding the use of atmospheric water‐related observations – especially at convective scales (also known as storm scales) – this article discusses the observation and assimilation of water‐related quantities. Special emphasis is placed on background error statistics for variational and hybrid methods which need special attention for water variables. The challenges of convective‐scale data assimilation of atmospheric water information are discussed, which are more difficult to tackle than at larger scales. Some of the most important challenges include the greater degree of inhomogeneity and lower degree of smoothness of the flow, the high volume of water‐related observations (e.g. from radar, microwave and infrared instruments), the need to analyse a range of hydrometeors, the increasing importance of position errors in forecasts, the greater sophistication of forward models to allow use of indirect observations (e.g. cloud‐ and precipitation‐affected observations), the need to account for the flow‐dependent multivariate “balance” between atmospheric water and both dynamical and mass fields, and the inherent non‐Gaussian nature of atmospheric water variables.

A dynamical Gaussian, lognormal, and reverse lognormal Kalman filter

Van Loon,

Fletcher

2023

We derive a generalization of the Kalman filter that allows for non‐Gaussian background and observation errors. The Gaussian assumption is replaced by considering that the errors come from a mixed distribution of Gaussian, lognormal, and reverse lognormal random variables. We detail the derivation for reverse lognormal errors, and extend the results to mixed distributions, where the number of Gaussian, lognormal, and reverse lognormal state variables can dynamically change every analysis time. We robustly test the dynamical mixed Kalman filter on two different systems based on the Lorenz 1963 model, and demonstrate that non‐Gaussian techniques generally improve the analysis skill if the observations are sparse and uncertain, compared to the Gaussian Kalman filter.This article is protected by copyright. All rights reserved.