Ensemble propagation and continuous matrix factorization algorithms

Bergemann, Kay; Gottwald, Georg A.; Reich, Sebastian

doi:10.1002/qj.457

Cited by 17 publications

(26 citation statements)

References 28 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular we use the method proposed in Tippett et al (2003) and Wang et al (2004), the so-called ensemble transform Kalman filter (ETKF). Alternatively one could have chosen the ensemble adjustment filter (Anderson, 2001) or the continuous Kalman-Bucy filter, which does not require the inversion of matrix inverses (Bergemann et al, 2009). A new forecast Z(t i+1 − ) is then obtained by propagating Z a with the full nonlinear dynamics to the next time of observation.…”

Section: Ensemble Kalman Filtermentioning

confidence: 99%

A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks

Gottwald

Majda

2013

Nonlin. Processes Geophys.

View full text Add to dashboard Cite

Abstract. We study catastrophic filter divergence in data assimilation procedures whereby the forecast model develops severe numerical instabilities leading to a blow-up of the solution. Catastrophic filter divergence can occur in sparse observational grids with small observational noise for intermediate observation intervals and finite ensemble sizes. Using a minimal five-dimensional model, we establish that catastrophic filter divergence is a numerical instability of the underlying forecast model caused by the filtering procedure producing analyses which are not consistent with the true dynamics, and stiffness caused by the fast attraction of the inconsistent analyses towards the attractor during the forecast step.

show abstract

Section: Ensemble Kalman Filtermentioning

confidence: 99%

A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks

Gottwald

Majda

2013

Nonlin. Processes Geophys.

View full text Add to dashboard Cite

show abstract

“…The ensemble based extension of the Kalman-Bucy filter [13] was introduced by Bergemann et al [37]. In this approach, the analysis step is formulated in terms of ordinary differential equations (ODEs).…”

Section: Ensemble Based Kalman-bucy Filtersmentioning

confidence: 99%

“…In this scheme, the analyzed ensemble is directly obtained as a solution of the ODEs over a time step. Amezcua et al [39] provide a discussion on the above approaches [37,38]. They point out that in both of these approaches the stability of the filter depends on the ratio of the forecast error covariance to the observation error covariance.…”

Section: Ensemble Based Kalman-bucy Filtersmentioning

confidence: 99%

See 1 more Smart Citation

Distributed ensemble Kalman filtering

Shahid

Üstebay

Coates

2014

2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM)

View full text Add to dashboard Cite

Distributed estimation in a wireless sensor network has many advantages. It eliminates the need of the centralized knowledge of the measurement model parameters and does not have a single point of failure. Also, any sensor node can be queried to retrieve the state estimate. Furthermore, for high dimensional measurements, local processing of information also results in a significant reduction in the communication overhead.The existing distributed Kalman filtering schemes have good computational and communication efficiency, but do not work well for non-linear and non-Gaussian problems. On the other hand, the distributed particle filtering schemes handle the non-linearities well, but have a much higher computational and communication cost.In this thesis, we propose novel distributed filtering schemes based on the ensemble Kalman filter (EnKF). We consider three forms of the EnKF and express their update equations in an alternative information form. This allows us to use the randomized gossip algorithm to reach consensus on the sufficient statistics and perform local updates. The simulation results show that all three forms of the EnKF have a considerably lower computational cost compared to the particle filters. The results suggest that the proposed distributed schemes outperform the existing state-of-the-art distributed filtering schemes in two scenarios, a) linear measurement model with non-linear state dynamics, and b) high dimensional measurements (model parameters known everywhere in the network) with nonlinear measurement model and state dynamics. In both of these scenarios, the proposed schemes achieve an estimation accuracy comparable to the existing state-of-the-art schemes while significantly reducing the communication cost.ii Résumé L'estimation distribuée dans un réseau de capteurs sans fil possède plusieurs avantages. Ellé elimine le besoin d'une connaissance centralisée des paramtres du modèle de mesure et n'a pas un point de défaillance unique. Aussi, n'importe quel agent-capteur peut-être consulté pour obtenir une approximation de l'état général. De plus, pour des mesures de hautes dimensions, le calcul local d'informations résulte en une réduction significative des coûts de communication.Les implémentations courantes du filtre de Kalman sont efficaces sur les plans de la charge de calcul et de la communication mais ne le sont pas pour les problmes non-linéaires et non-gaussiens. D'un autre côté, les techniques distribuées de filtrage particulaire gèrent avec succès les cas non-linéaires mais sont coûteuses sur les plans de la charge de calcul et de la communication.Dans cette thèse, nous proposons des techniques de filtrage distribué basées sur le filtre de Kalman d'ensemble (FKEn). Nous considérons trois formes du FKEn et exprimons leurséquations de changement sous une forme alternative. Cela nous permet d'utiliser un algorithme de gossip aléatoire afin d'atteindre un consensus sur les statistiques suffisantes et calculer les changements locaux. Les résultats des simulations montrent que ...

show abstract

“…Alternatively one could choose the ensemble adjustment filter (Anderson, 2001) in which the ensemble deviation matrix Z f is pre-multiplied with an appropriately determined matrix A ∈ R N×N . Yet another method to construct analysis deviations was proposed in Bergemann et al (2009) where the continuous Kalman-Bucy filter is used to calculate Z a without using any computations of matrix inverses, which may be advantageous in high-dimensional systems. A new forecast is obtained by propagating Z a with the nonlinear forecast model to the next observation time, where a new analysis cycle will be started.…”

Section: The Variance-limiting Kalman Filtermentioning

confidence: 99%

Controlling model error of underdamped forecast models in sparse observational networks using a variance‐limiting Kalman filter

Mitchell

Gottwald

2012

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

The problem of controlling covariance overestimation due to underdamped forecast models and sparsity of the observational network in an ensemble Kalman filter setting is considered. It is shown in a variational setting that limiting the analysis-error covariance to stay below the climatological value and driving the mean towards the climatological mean for the unobserved variables can significantly improve analysis skill over standard ensemble Kalman filters. These issues are explored for a Lorenz-96 system. It is shown that, for large observation intervals, the climatological information assures that the statistical properties of the unobserved variables are recovered, providing superior analysis skill. This skill improvement is increased for larger observational noise. Copyright c 2012 Royal Meteorological SocietyKey Words: data assimilation; ensemble Kalman filter; Lorenz-96; model error

show abstract

Ensemble propagation and continuous matrix factorization algorithms

Cited by 17 publications

References 28 publications

A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks

A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks

Distributed ensemble Kalman filtering

Controlling model error of underdamped forecast models in sparse observational networks using a variance‐limiting Kalman filter

Contact Info

Product

Resources

About