Irreversible Monte Carlo algorithms for efficient sampling

Abstract

Equilibrium systems evolve according to Detailed Balance (DB). This principe guided development of the Monte-Carlo sampling techniques, of which Metropolis-Hastings (MH) algorithm is the famous representative. It is also known that DB is sufficient but not necessary. We construct irreversible deformation of a given reversible algorithm capable of dramatic improvement of sampling from known distribution. Our transformation modifies transition rates keeping the structure of transitions intact. To illustrate the general scheme we design an Irreversible version of Metropolis-Hastings (IMH) and test it on example of a spin cluster. Standard MH for the model suffers from the critical slowdown, while IMH is free from critical slowdown. [3,4,5] for a sample set of reviews in physics and computer science.) If one formally follows the letter of the original MCMC suggestion one ought to ensure that the Detailed Balance (DB) condition is satisfied. This condition reflects microscopic reversibility of the underlying equilibrium dynamics. A reader, impressed with indisputable success of the reversible MCMC techniques, may still wonder if the equilibrium dynamics is the most efficient strategy for sampling and evaluating the integrals? In this letter we argue that typically the answer is NO. Let us try to illustrate the ideas on a simple everyday life example. Consider mixing sugar in a cup of coffee, which is similar to sampling, as long as the sugar particles have to explore the entire interior of the cup. DB dynamics corresponds to diffusion taking an enormous mixing time. This is certainly not the best way to mix. Moreover, our everyday experience suggests a better solutionenhance mixing with a spoon. Spoon steering generates an out-of-equilibrium external flow which significantly accelerates mixing, while achieving the same final result -uniform distribution of sugar concentration over the cup. In this letter we show constructively, with a practical algorithm suggested, that similar strategy can be used to decrease mixing time of any known reversible MCMC algorithms.There are two main obstacles which prevent fast mixing by traditional MCMC methods. First, the effective energy landscape can have high barriers, separating the energy minima. In this case mixing time is dominated by rare processes of overcoming the barriers. Second, slow mixing can originate from the high entropy of the states basin (too many comparably important states) providing major contribution to the system partition function. In the later case mixing time is determined by the number of steps it takes for reversible (diffusive) random walk to explore all the relevant states. MCMC algorithms are best described on discrete example. Consider a graph with vertices i = 1, . . . , N each labeling a state of the system and edges (i → j) corresponding to "allowed" transitions between the states. For instance, an Ndimensional hypercube corresponds to a system of N spins (with N = 2 N states) with single-spin flips allowed. An MCMC algorithm can be described in ...

Non-reversible Metropolis-Hastings

“…This can be shown experimentally in special cases Suwa and Todo (2010), Turitsyn et al (2011), Vucelja (2014), theoretically in special cases Diaconis et al (2000), Neal (2004), and in fact, also in general Sun et al (2010), Chen and Hwang (2013), with respect to asymptotic variance. See also Rey-Bellet and Spiliopoulos (2014) for improved asymptotic variance of non-reversible diffusion processes on compact manifolds.…”

“…There exist two basic approaches to the construction of non-reversible chains from reversible chains: one can 'lift' the Markov chain to a larger state space Diaconis et al (2000), Neal (2004), Turitsyn et al (2011), Vucelja (2014), or one can introduce non-reversibility without altering the state space Sun et al (2010). In continuous spaces, the hybrid or Hamiltonian Monte Carlo Horowitz (1991), Neal (2011) is closely related to the lifting approach in discrete spaces.…”

Dynamics of One-Dimensional Ising Model without Detailed Balance Condition

“…In this section, we construct a Markov chain for the onedimensional Ising model on a basis of skew detailed balance condition (SDBC) 4) and study time evolution of the magnetization density under SDBC.…”

“…3.1 Master equation and skew detailed balance condition According to the method of Turitsyn et al, 4) we introduce another Ising spin ε = ±1 in addition to the original spin configurations. The enlarged state of the system is denoted by X := (σ, ε) ∈ {−1, +1} N +1 .…”

“…MCMC methods which are not based on DBC have been discussed for improving the performance and, in fact, some MCMC methods without DBC have recently been proposed. [3][4][5] In the MCMC method, the convergence to the target distribution is guaranteed by using an irreducible and aperiodic Markov chain, characterized by a transition matrix. It is known that the rate of convergence of the distribution depends on the second largest eigenvalue of the transition matrix of the Markov chain.…”