Adaptive Mixture Modeling Metropolis Methods for Bayesian Analysis of Nonlinear State-Space Models

Abstract

We describe a strategy for Markov chain Monte Carlo analysis of non-linear, non-Gaussian state-space models involving batch analysis for inference on dynamic, latent state variables and fixed model parameters. The key innovation is a Metropolis-Hastings method for the time series of state variables based on sequential approximation of filtering and smoothing densities using normal mixtures. These mixtures are propagated through the non-linearities using an accurate, local mixture approximation method, and we use a regenerating procedure to deal with potential degeneracy of mixture components. This provides accurate, direct approximations to sequential filtering and retrospective smoothing distributions, and hence a useful construction of global Metropolis proposal distributions for simulation of posteriors for the set of states. This analysis is embedded within a Gibbs sampler to include uncertain fixed parameters. We give an example motivated by an application in systems biology. Supplemental materials provide an example based on a stochastic volatility model as well as MATLAB code.

A Comparison of Inferential Methods for Highly Nonlinear State Space Models in Ecology and Epidemiology

“…Notably, this work does not include MCMC methods for parameter identification, such as those proposed by Carlin, Polson and Stoffer (1992), Geweke and Tanizaki (2001), Polson, Stroud and Müller (2008) and Niemi and West (2010). One reason for this is that highly non-linear models, such as those considered here, are often characterized by strong dependencies between states and static parameters.…”

Generalized Dynamic Factor Models for Mixed-Measurement Time Series

“…Stroud et al (2003) further proposed to jointly update all unobserved states by using a Metropolis Hastings proposal that approximates the exact conditional posterior. Closer to our perspective is the recent work of Niemi and West (2010), who proposed adaptive mixture approximations by matching moments for the smoothing distributions in non-linear state-space models. However, most of the these algorithms are not designed for cases with multidimensional mixed-measurement response data, in which presence of multiple exponential family distributions of data makes sequential approximation of smoothing distributions more computationally challenging, especially when missing data exist.…”

“…Efficient adaptive Metropolis Hastings via GDKA to sample time-varying latent factors is one of the key steps in our sampling scheme, and it is well known that the acceptance rates of Metropolis Hastings steps tend to decrease rapidly as the number of time points increases (Niemi and West (2010)). To evaluate this, we generated 100 simulated datasets for each T and look at the overall performance.…”

Bayesian Analysis of Latent Threshold Dynamic Models

“…This follows related approaches using this idea of a global FFBS-based proposal (Prado and West 2010;Niemi and West 2010). However, the resulting acceptance rates decrease exponentially with T and some simulation experiments indicate unacceptably low acceptance rates in several examples, especially with higher levels of sparsity when the proposal distribution from the non-threshold model agrees less and less with the posterior under the LTM.…”