Information Geometry of Reversible Markov Chains

Abstract

We analyze the information geometric structure of time reversibility for parametric families of irreducible transition kernels of Markov chains. We define and characterize reversible exponential families of Markov kernels, and show that irreducible and reversible Markov kernels form both a mixture family and, perhaps surprisingly, an exponential family in the set of all stochastic kernels. We propose a parametrization of the entire manifold of reversible kernels, and inspect reversible geodesics. We define information projections onto the reversible manifold, and derive closed-form expressions for the e-projection and m-projection, along with Pythagorean identities with respect to information divergence, leading to some new notion of reversiblization of Markov kernels. We show the family of edge measures pertaining to irreducible and reversible kernels also forms an exponential family among distributions over pairs. We further explore geometric properties of the reversible family, by comparing them with other remarkable families of stochastic matrices. Finally, we show that reversible kernels are, in a sense we define, the minimal exponential family generated by the m-family of symmetric kernels, and the smallest mixture family that comprises the e-family of memoryless kernels.

Metropolis–Hastings reversiblizations of non-reversible Markov chains

“…(1) Generating reversiblizations via geometric projections. This approach continues the line of work initiated in Billera and Diaconis (2001); Diaconis and Miclo (2009); Wolfer and Watanabe (2021), in which reversiblizations are viewed as projections under information divergences such as f -divergences. The advantage of this approach is that we can recover all known reversiblizations in a unified framework.…”

“…It is instructive to note that our notions of projection are with respect to a fixed target π, while in Wolfer and Watanabe (2021) projections are onto the entire reversible set. In the context of Markov chain Monte Carlo, we are often given a target π for instance a posterior distribution in a Bayesian model, and in this setting it is not at all restrictive to consider and investigate projections onto L(π).…”

“…which are, respectively, the geometric mean reversiblization as studied in Diaconis and Miclo (2009); Wolfer and Watanabe (2021), M 2 -reversiblization as proposed in Choi (2020), and the classical Metropolis-Hastings reversiblization. We also call the case of p = 1/3 to be the Lorentz mean reversiblization as it is the Lorentz mean known in the literature Lin (1974).…”

“…Inspired by the graphs in Billera and Diaconis (2001); Choi and Huang (2020); Wolfer and Watanabe (2021) and to visualize the concept of centroid, we illustrate two f -projection centroids in a rectangle and in an eight-sided polygon in Figure 1. Similar graphs can be drawn for f * -projection centroids but with the direction of the arrows flipped.…”

“…More recently, higher order multiplicative reversiblizations are used to define a pseudo-spectral gap and develop concentration inequalities of non-reversible Markov chains (Paulin, 2015), while the second Metropolis-Hastings reversiblization is proposed and investigated in Choi (2020); Choi and Huang (2020). The geometric mean reversiblization is analyzed in Diaconis and Miclo (2009) in continuous-time and in Wolfer and Watanabe (2021) in discrete-time, while the Barker proposal (which is in fact a harmonic mean reversiblization, see Section 3.3 and Section 5 below) has recently enjoyed considerable interest in the Markov chain Monte Carlo literature in a series of papers Vogrinc et al, 2022;Zanella, 2020).…”