Importance sampling for partially observed temporal epidemic models

Abstract

We present an importance sampling algorithm that can produce realisations of Markovian epidemic models that exactly match observations, taken to be the number of a single event type over a period of time. The importance sampling can be used to construct an efficient particle filter that targets the states of a system and hence estimate the likelihood to perform Bayesian inference. When used in a particle marginal Metropolis Hastings scheme, the importance sampling provides a large speed-up in terms of the effective sample size per unit of computational time, compared to simple bootstrap sampling. The algorithm is general, with minimal restrictions, and we show how it can be applied to any continuous-time Markov chain where we wish to exactly match the number of a single event type over a period of time.

The Causal Effects of Lockdown Policies on Health and Macroeconomic Outcomes

“…The state-of-the-art methods for Bayesian analysis of infectious diseases are reviewed by Alahmadi et al (2020) and Broemeling (2021). New research in the area includes approximate Bayesian Computation for epidemic models by Kypraios et al (2017) and Minter and Retkute (2019), importance sampling-based Bayesian inference by Black (2019) and Li et al (2020). That last paper, the closest to our approach, uses an ensemble adjustment Kalman filter to estimate with maximum likelihood an epidemiological model for 375 Chinese cities.…”

Direct statistical inference for finite Markov jump processes via the matrix exponential

“…Particle MCMC and SMC, however, are relatively complex algorithms, even more so when a bootstrap particle filter (simulation from the process itself) is not suitable and a bridge simulator is necessary, such as when observation noise is small or when there is considerable variability in the state from one observation to the next; see Golightly and Wilkinson (2015), Golightly and Sherlock (2019), Black (2019). In cases where the number of states, d, is finite, direct exact likelihood-based inference is available via the exponential of the infinitesimal generator for the continuous-time Markov chain, or rate matrix, Q. Whilst such inference is conceptually straightforward, it has often been avoided in practice for general MJPs, except in cases where the number of states is very small e.g.…”

Bayesian model discrimination for partially-observed epidemic models

“…Our method works by calculating unbiased estimates of the likelihood via sequential importance resampling, which in turn is used in another importance sampling algorithm to estimate posterior model probabilities. A novel feature of this method is that the likelihood is estimated via a scheme which is ideally suited for partially-observed CTMCs, where one component of the state is observed exactly [8]. This works by sampling realisations of the partially-observed process in a way that realisations always match with observations.…”

“…In this paper we use importance sampling in the space of latent variables to estimate likelihoods via sequential importance resampling and within the parameter space to calculate the model evidence. The novelty of this approach is that the likelihood is estimated via the most suitable kind of importance sampling scheme for these kinds of models, as described in [8].…”

“…The difficulty in evolving particles in the cases considered here is that precise observations are made from models with many latent variables, which can make data-augmentation or rejection-sampling approaches slow. In Sections 3.2 and 4.3 we describe the method from [8], which builds upon [26], for generating realisationsx (t−1,t] from initial state x t−1 for each of the case studies. The main benefit of this approach is that we use an efficient sampling distribution q(•|y t , x t−1 ) which is tractable and generatesx (t−1,t] which match observations almost surely, and, are similar to the partially-observed process.…”