On the stability of sequential Monte Carlo methods in high dimensions

Abstract

We investigate the stability of a Sequential Monte Carlo (SMC) method applied to the problem of sampling from a target distribution on R d for large d. It is well known [9,14,56] that using a single importance sampling step one produces an approximation for the target that deteriorates as the dimension d increases, unless the number of Monte Carlo samples N increases at an exponential rate in d. We show that this degeneracy can be avoided by introducing a sequence of artificial targets, starting from a 'simple' density and moving to the one of interest, using an SMC method to sample from the sequence (see e.g. [20,27,38,48]). Using this class of SMC methods with a fixed number of samples, one can produce an approximation for which the effective sample size (ESS) converges to a random variable εN as d → ∞ with 1 < εN < N . The convergence is achieved with a computational cost proportional to N d 2 . If εN ≪ N , we can raise its value by introducing a number of resampling steps, say m (where m is independent of d). In this case, ESS converges to a random variable εN,m as d → ∞ and limm→∞ εN,m = N . Also, we show that the Monte Carlo error for estimating a fixed dimensional marginal expectation is of order 1 √ N uniformly in d. The results imply that, in high dimensions, SMC algorithms can efficiently control the variability of the importance sampling weights and estimate fixed dimensional marginals at a cost which is less than exponential in d and indicate that, in high dimensions, resampling leads to a reduction in the Monte Carlo error and increase in the ESS.

Sequentially Constrained Monte Carlo

“…The length of the sequence must be tuned to the dimensionality of the problem for SMC to be stable (Beskos et al, 2014). The two main SMC sequences of bridging distributions are based on gradually introducing the likelihood in the posterior.…”

A local particle filter for high-dimensional geophysical systems

“…Applying such an approach, van Leeuwen (2003) considers a model for the Agulhas Current with dimension of roughly 2 × 10 5 . Further, Beskos et al (2012) discuss recursive methods for estimating the proposal densities, similar to the running-in-place algorithm (Yang et al, 2012a, b; that has been used with LETKF in meteorological and oceanographic data assimilation. Xiong et al (2006) presented techniques related to the ensemble transform Kalman filter (ETKF) that may provide an alternative approach to the common SIR method for generating particle estimates for the posterior distribution.…”

Particle Filters for nonlinear data assimilation in high-dimensional systems

“…Furthermore, as pointed out by the authors and earlier practitioners, like Bengtsson et al [4] and Van Leeuwen [13], a fundamental problem is that the discontinuous boundaries can lead to unrealistic model states, potentially ruining the model forecasts. Another interesting recent work in this area is Beskos et al [5], who discuss the convergence rate a particle filter that updates a sequence of artificial targets, and these targets can be chosen as marginal posterior pdf's, making a direct connection with localisation.…”