Particle filters and Markov chains for learning of dynamical systems

Abstract

Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods provide computational tools for systematic inference and learning in complex dynamical systems, such as nonlinear and non-Gaussian state-space models. This thesis builds upon several methodological advances within these classes of Monte Carlo methods.Particular emphasis is placed on the combination of SMC and MCMC in so called particle MCMC algorithms. These algorithms rely on SMC for generating samples from the often highly autocorrelated state-trajectory. A specific particle MCMC algorithm, referred to as particle Gibbs with ancestor sampling (PGAS), is suggested. By making use of backward sampling ideas, albeit implemented in a forward-only fashion, PGAS enjoys good mixing even when using seemingly few particles in the underlying SMC sampler. This results in a computationally competitive particle MCMC algorithm. As illustrated in this thesis, PGAS is a useful tool for both Bayesian and frequentistic parameter inference as well as for state smoothing. The PGAS sampler is successfully applied to the classical problem of Wiener system identification, and it is also used for inference in the challenging class of non-Markovian latent variable models.Many nonlinear models encountered in practice contain some tractable substructure. As a second problem considered in this thesis, we develop Monte Carlo methods capable of exploiting such substructures to obtain more accurate estimators than what is provided otherwise. For the filtering problem, this can be done by using the well known RaoBlackwellized particle filter (RBPF). The RBPF is analysed in terms of asymptotic variance, resulting in an expression for the performance gain offered by Rao-Blackwellization. Furthermore, a Rao-Blackwellized particle smoother is derived, capable of addressing the smoothing problem in so called mixed linear/nonlinear state-space models. The idea of Rao-Blackwellization is also used to develop an online algorithm for Bayesian parameter inference in nonlinear state-space models with affine parameter dependencies.v Populärvetenskaplig sammanfattningMatematiska modeller av dynamiska förlopp används inom i stort sett alla tekniska och naturvetenskapliga discipliner. Till exempel, inom epidemiologi används modeller för att prediktera, dvs. förutsäga, spridningen av influensavirus inom en population. Antag att vi gör regelbundna observationer av hur många personer i populationen som är smittade. Baserat på denna information kan en modell användas för att prediktera antalet nya sjukdomsfall under, låt säga, nästkommande veckor. Den här typen av information möjliggör att en epidemi kan identifieras i ett tidigt skede, varpå åtgärder kan tas för att minska dess påverkan. Ett annat exempel är att prediktera hur hastigheten och orienteringen på ett flygplan påverkas då en viss styrsignal ställs ut på rodren, vilket är viktigt vid styrsystemdesign. Sådana prediktioner kräver en modell av flygplanets dynamik. Ytterligare ett exempel är att prediktera utvecklingen på e...

Comparison of simulation-based algorithms for parameter estimation and state reconstruction in nonlinear state-space models

“…this algorithm generally has a poor mixing and a low rate of convergence. The main reason for this is the so-called degeneracy issue [33]: all the particles present at the final time step T share the same ancestors after a few generations. This is illustrated on Figure 4 where all the particles present at time t = 4 have the same grandparent at time t = 2.…”

“…The resulting algorithm is referred to as Conditional Particle Filtering-Ancestor Sampling (CPFAS) in the sequel. In [33,34], it is shown empirically that this algorithm is efficient to simulate trajectories of the smoothing distribution with only 5 − 20 particles. It is also proven that theoretical properties of the original CPF algorithm hold true for the CPFAS (see Theorem A in Appendix).…”

“…However, a large amount of particles and thus simulations of the dynamical model is typically required to get good approximations of the smoothing distribution. Since 2010, conditional particle filters (CPFs) [34,36,33,44] pioneered by [1] have been developed as an alternative strategy to approximate smoothing distributions at a lower computational cost. Contrary to the more usual particle smoothing algorithms discussed above, CPFs simulate samples of the smoothing distribution using an iterative algorithm.…”

“…The authors showed using numerical simulations of univariate toy models that the algorithm can estimate the variances Q and R using only a few particles. However, CPFAS suffers from degeneracy (see [33]) and consequently the estimators obtained with CPFAS-SEM may be biased and/or have large variance.…”

Frequency tracking with intermittent wrapped phase measurement using the Rao-Blackwellized particle filter

“…The advantage of combining the two methods is that the dimension of the state that needs to be approximated by the particle filter is decreased allowing for far fewer particles required to cover the entire state space. The reader is referred to [9] for a more detailed description of the RBPF.…”