Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo

Abstract

Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample from, say, a posterior distribution. Interestingly, continuous-time algorithms seem particularly well suited to Bayesian analysis in big-data settings as they need only access a small sub-set of data points at each iteration, and yet are still guaranteed to target the true posterior distribution. Whilst continuous-time MCMC and SMC methods have been developed independently we show here that they are related by the fact that both involve simulating a piecewise deterministic Markov process. Furthermore we show that the methods developed to date are just specific cases of a potentially much wider class of continuous-time Monte Carlo algorithms. We give an informal introduction to piecewise deterministic Markov processes, covering the aspects relevant to these new Monte Carlo algorithms, with a view to making the development of new continuous-time Monte Carlo more accessible. We focus on how and why sub-sampling ideas can be used with these algorithms, and aim to give insight into how these new algorithms can be implemented, and what are some of the issues that affect their efficiency.

Constructing Sampling Schemes via Coupling: Markov Semigroups and Optimal Transport

“…associated to the decomposition (33). Applying Lemma 18, we see immediately that the first condition of Definition 6 is satisfied.…”

“…Note however that the decomposition (33) is not unique, since there are (infinitely) many ways of choosing the operators A i k . A particular choice of decomposing the generators L i as in (33) hence essentially amounts to the choice of squareroots for the symmetric parts. We remark here that naturally C ∞ c (Ē) ⊂ D(A i k ) and C ∞ c (Ē) ⊂ D(B i ) are implicitly assumed, authorising the computations in later sections.…”

“…where the adjoint is taken in L 2 (π). In particular, L can naturally be written in the form (33), with A = ∂ x and B = 0. To illustrate trivial couplings (see Example 10), consider first two independent identical copies of (42), denoted by (X t , Y t ) t≥0 , hence evolving according to the dynamics…”

Coordinate sampler: a non-reversible Gibbs-like MCMC sampler

“…In practice, the existing PDMP-based samplers choose V to be the Euclidean space R d , the sphere S d−1 , or the discrete set VV == {v = (v 1 , · · · , v d )|v i ∈ {−1, 1}, i = 1, · · · , d}. Following Fearnhead et al (2018), a piecewise deterministic Markov process Z t = (X t , V t ) consists of three distinct components: a deterministic dynamic between events, an event occurrence rate, and a transition dynamic at event times. Specifically, 1.…”

“…Bierkens et al (2018) considers the application of PDMP for distributions on restricted domains. Fearnhead et al (2018) unify BPS and Zigzag samplers within the framework of PDMPs: they propose a choice of the process velocity, at event times, over the unit sphere, based on the angle between this velocity and the gradient of the potential function. (This perspective relates to the transition dynamics used here.)…”