Particle Filters and Data Assimilation

Abstract

State-space models can be used to incorporate subject knowledge on the underlying dynamics of a time series by the introduction of a latent Markov state-process. A user can specify the dynamics of this process together with how the state relates to partial and noisy observations that have been made. Inference and prediction then involves solving a challenging inverse problem: calculating the conditional distribution of quantities of interest given the observations. This article reviews Monte Carlo algorithms for solving this inverse problem, covering methods based on the particle filter and the ensemble Kalman filter. We discuss the challenges posed by models with high-dimensional states, joint estimation of parameters and the state, and inference for the history of the state process. We also point out some potential new developments which will be important for tackling cutting-edge filtering applications.Comment: To appear in `Annual Review of Statistics and Its Application

Elements of Sequential Monte Carlo

“…Several overview and tutorial articles focus on particle filters, i.e. the SMC algorithms specifically tailored to solve the online filtering problem [Arulampalam et al, 2002, Doucet and Johansen, 2009, Fearnhead and Künsch, 2018. In this tutorial we will take a different view and explain how SMC can be used to solve more general "offline" problems.…”

Iterative near-term forecasting of the transmission and management of SARS-CoV-2/COVID-19 using social interventions at the county-level in the United States

“…the fraction of asymptomatic infecteds) would offer better constraining of parameters and hence forecast variance. However, this has to be balanced by appropriately addressing the effects of parameter degeneracy and sample impoverishment which would impact the ability of the model to t novel data as transmission conditions change drastically over the near future 28,[51][52][53] . We have used a resampling approach whereby at each sequential updating point, we have blended in 25% random samples from initial priors to the posteriors obtained during the uptake made a time step (every 2-weeks) previously to keep forecast error below 20% to address this problem in the simulations reported here.…”

“…Based on this performance metric, the best-tting 500 parameter vectors are retained as the most likely parameter sets to describe the local outbreak during the chosen 14-day window. For simulating the epidemic for the next 14 day period, another 50,000 parameters sets are sampled of which 75% are randomly sampled from the posterior distribution of the most recent 14 day window, while another 25% are sampled from the initial parameter priors to avoid sample depletion 52,53 . These set of blended parameter vectors are used to sequentially select the best-tting models over time, and are used to forecast the impacts of the interventions described above.…”

Review of State-Space Models for Fisheries Science

“…Kantas et al, 2015). We defer to Doucet, de Freitas, and Gordon (2001) for an overview of the evolution of importance sampling into sequential importance sampling and the advent of modern sequential Monte Carlo methods, which are also known as particle filters (see Fearnhead and Künsch, 2018); see also Hürzeler and Künsch (2001) and Liu and West (2001). That said, variants of MCMC were developed in parallel; the ones based on Hamiltonian dynamics (see Neal, 2011) proved to be highly efficient with SSMs and are available in contemporary MCMC software such as Stan (Stan Development Team, 2016).…”