The GIGG‐EnKF: ensemble Kalman filtering for highly skewed non‐negative uncertainty distributions

Bishop, Craig H.

doi:10.1002/qj.2742

Cited by 47 publications

(71 citation statements)

References 69 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been shown that these factors in ensemble systems can result in a suboptimal estimate of the state [ Bishop , ]. Methods are currently being considered for dealing with near‐background aerosol levels in the ENAAPS‐DART system [ Hodyss , ; Bishop , ] as well as bias correction in order to improve performance in the ensemble. In comparison to the EAKF, the 2DVar was able to better deal with near‐background aerosol levels, resulting in smaller analysis errors in locations where AOT levels are near or below 0.1. The 24 h AOT forecast errors were evaluated in both systems with the inclusion of AERONET.…”

mentioning

confidence: 99%

Assimilation of AERONET and MODIS AOT observations using variational and ensemble data assimilation methods and its impact on aerosol forecasting skill

Rubin

Reid²,

Hansen³

et al. 2017

JGR Atmospheres

View full text Add to dashboard Cite

Data assimilation of Aerosol Robotic Network (AERONET) and Moderate Resolution Imaging Spectroradiometer (MODIS) aerosol optical thickness (AOT) for aerosol forecasting was tested within the Navy Aerosol Analysis Prediction System (NAAPS) framework, using variational and ensemble data assimilation methods. Navy aerosol forecasting currently makes use of a deterministic NAAPS simulation coupled to Navy Variational Data Assimilation System for aerosol optical depth, a two‐dimensional variational data assimilation system, for MODIS AOT assimilation. An ensemble version of NAAPS (ENAAPS) coupled to an ensemble adjustment Kalman filter (EAKF) from the Data Assimilation Research Testbed was recently developed, allowing for a range of data assimilation and forecasting experiments to be run with deterministic NAAPS and ENAAPS. The main findings are that the EAKF, with its flow‐dependent error covariances, makes better use of sparse observations such as AERONET AOT. Assimilating individual AERONET observations in the two‐dimensional variational system can increase the analysis errors when observations are located in high AOT gradient regions. By including AERONET with MODIS AOT assimilation, the magnitudes of peak aerosol events (AOT > 1) were better captured with improved temporal variability, especially in India and Asia where aerosol prediction is a challenge. Assimilating AERONET AOT with MODIS had little impact on the 24 h forecast skill compared to MODIS assimilation only, but differences were found downwind of AERONET sites. The 24 h forecast skill was approximately the same for forecasts initialized with analyses from AERONET AOT assimilation alone compared to MODIS assimilation, particularly in regions where the AERONET network is dense; including the United States and Europe, indicating that AERONET could serve as a backup observation network for over‐land synoptic‐scale aerosol events.

show abstract

mentioning

confidence: 99%

Assimilation of AERONET and MODIS AOT observations using variational and ensemble data assimilation methods and its impact on aerosol forecasting skill

Rubin

Reid²,

Hansen³

et al. 2017

JGR Atmospheres

View full text Add to dashboard Cite

show abstract

“…Bishop () introduced a new filter that properly and efficiently accounts for special cases of NG variables, namely data with gamma (Gam) or inverse gamma (IG) PDFs in the way described below. Forms of the Gam ( p Gam ) and IG ( p IG ) PDFs are as follows,

p_{normalGam} (q) = \frac{{(q σ_{normalr}^{- 2} / \overset{true‾}{q})}^{σ_{normalr}^{- 2}}}{Γ (σ_{r}^{- 2}) q} \exp (- σ_{normalr}^{- 2} q / \overset{true‾}{q}),

p_{normalIG} (q) = \frac{{({\overset{σ}{true}}_{normalr}^{- 2} \overset{true‾}{q})}^{({\overset{σ}{true}}_{normalr}^{- 2} + 1)}}{Γ ({\overset{σ}{true}}_{normalr}^{- 2} + 1) q^{((), {\overset{true˜}{σ}}_{r}^{- 2} + 2)}} \exp (- {\overset{σ}{true}}_{normalr}^{- 2} \overset{true‾}{q} / q),

where

\overset{true‾}{q}

is the mean of each respective PDF and

σ_{normalr}^{2}

and

{\overset{σ}{true}}_{normalr}^{2}

are the type I and II relative variances respectively.…”

Section: Non‐gaussianitymentioning

confidence: 99%

“…The textbook definitions of these PDFs are usually written in terms of shape and scale parameters (e.g. for Gam,

σ_{normalr}^{- 2}

is the shape parameter and

\overset{true‾}{q} / σ_{normalr}^{- 2}

is the scale parameter), but Bishop () shows how using the above two types of relative variance allows some useful properties to be revealed when Gam/IG PDFs are used as the prior and likelihood in Bayes' Theorem for estimating a scalar quantity with a single observation. These properties are as follows.…”

Section: Non‐gaussianitymentioning

confidence: 99%

“…Note that the likelihood functions are found from Equation 27 or 28 by using the observed value as the random variable ( q → q o ), and then the possible truth as the mean (

\overset{true‾}{q} \to q

). Bishop () used these properties to develop a stochastic ensemble filter (the GIGG‐EnKF) to properly represent the error statistics of quantities that have Gam and IG statistics in the above manner. GIGG stands for Gamma‐Inverse‐Gamma‐Gaussian (Bishop's scheme also allows for Gaussian PDFs).…”

Section: Non‐gaussianitymentioning

confidence: 99%

See 1 more Smart Citation

Techniques and challenges in the assimilation of atmospheric water observations for numerical weather prediction towards convective scales

Bannister

Chipilski

Martínez‐Alvarado

2019

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

While contemporary numerical weather prediction models represent the large‐scale structure of moist atmospheric processes reasonably well, they often struggle to maintain accurate forecasts of small‐scale features such as convective rainfall. Even though high‐resolution models resolve more of the flow, and are therefore arguably more accurate, moist convective flow becomes increasingly nonlinear and dynamically unstable. Importantly, the models' initial conditions are typically sub‐optimal, leaving scope to improve the accuracy of forecasts with improved data assimilation. To address issues regarding the use of atmospheric water‐related observations – especially at convective scales (also known as storm scales) – this article discusses the observation and assimilation of water‐related quantities. Special emphasis is placed on background error statistics for variational and hybrid methods which need special attention for water variables. The challenges of convective‐scale data assimilation of atmospheric water information are discussed, which are more difficult to tackle than at larger scales. Some of the most important challenges include the greater degree of inhomogeneity and lower degree of smoothness of the flow, the high volume of water‐related observations (e.g. from radar, microwave and infrared instruments), the need to analyse a range of hydrometeors, the increasing importance of position errors in forecasts, the greater sophistication of forward models to allow use of indirect observations (e.g. cloud‐ and precipitation‐affected observations), the need to account for the flow‐dependent multivariate “balance” between atmospheric water and both dynamical and mass fields, and the inherent non‐Gaussian nature of atmospheric water variables.

show abstract

“…Bishop () introduced a variation on the perturbed observations form of the EnKF that enables it to utilize gamma, inverse‐gamma as well as Gaussian uncertainties. This Gamma, Inverse‐Gamma, and Gaussian (GIGG)‐EnKF filter accommodates positive‐definite variables, as well as observation error standard deviations that scale with the true value of the observed variable.…”

Section: Introductionmentioning

confidence: 99%

Nonlinear data assimilation for clouds and precipitation using a gamma inverse‐gamma ensemble filter

Posselt

Bishop

2018

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

Where clouds occur, their water content is always positive definite, and may be near zero. In addition, it is common for errors in remote‐sensing observations of clouds and rainfall to be represented as a fraction of the measurement. Furthermore, there is nonlinearity in the relationships among cloud environment, cloud microphysical processes, and the amount and distribution of cloud and precipitation. For these reasons, data assimilation algorithms that rely on linearity and assumptions of Gaussian probability distributions may have difficulty in assimilating observations in cloudy regions, as well as producing an analysis that realistically represents the actual distribution of clouds and precipitation. A recently developed ensemble filter algorithm, the Gamma, Inverse‐Gamma, and Gaussian Ensemble Kalman Filter (GIGG‐EnKF), allows for fractional observation errors and positive‐definite quantities. As such, it has promise for producing more effective and accurate data assimilation and retrievals of clouds and precipitation. This study evaluates the effectiveness of the GIGG‐EnKF by using it to estimate the values of warm rain cloud microphysical parameters in a cloud model using observations of precipitation. GIGG‐EnKF estimates of cloud parameters and rainfall are compared with a Bayesian reference solution returned by a Markov chain Monte Carlo algorithm, and with a perturbed‐observations Ensemble transform Kalman filter (ETKF). The ETKF produces an ensemble with a substantial number of negative (non‐physical) precipitation rates and parameter values. In contrast, the GIGG‐EnKF precipitation analysis is positive definite, and is able to adapt to changes in the observation value (and observation uncertainty). Nonlinearity in the parameter–precipitation relationship is handled by implementing an iterative outer loop regression from the observation space to state space. The positive‐definite constraint and natural accommodation of fractional error, along with the iterative outer loop, produces an accurate reproduction of the Bayesian analysis of the precipitation rate and cloud microphysical parameter values.

show abstract

The GIGG‐EnKF: ensemble Kalman filtering for highly skewed non‐negative uncertainty distributions

Cited by 47 publications

References 69 publications

Assimilation of AERONET and MODIS AOT observations using variational and ensemble data assimilation methods and its impact on aerosol forecasting skill

Assimilation of AERONET and MODIS AOT observations using variational and ensemble data assimilation methods and its impact on aerosol forecasting skill

Techniques and challenges in the assimilation of atmospheric water observations for numerical weather prediction towards convective scales

Nonlinear data assimilation for clouds and precipitation using a gamma inverse‐gamma ensemble filter

Contact Info

Product

Resources

About