Occupation laws for some time-nonhomogeneous Markov chains

Abstract

We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/n ζ where G is a "generator" matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = − k =i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit some different, perhaps unexpected, occupation behaviors depending on parameters.Although it is shown, on the one hand, that the position at time n converges to a point-mixture for all ζ > 0, on the other hand, the average occupation vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, is seen to converge to a constant, a pointmixture, or a distribution µ G with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of "spreading" between the cases 0 < ζ < 1 and ζ > 1.In particular, when G is appropriately chosen, intriguingly, µ G is a Dirichlet distribution, reminiscent of results in Pólya urns.

Fluctuations of the empirical measure of freezing Markov chains

“…In this section, we consider a particular case of freezing Markov chain, where all the states are connected, and the jump rate to a state does not depend on the position of the chain. This example of Markov chain has already been studied in the literature, for instance in [DS07]. Section 4.1 deals with the general D-dimensional case, for which most of the results of Section 3 can be written explicitly, notably the invariant measure of the exponential zig-zag process, which is a mixture of Dirichlet distributions (see Figure 4.1).…”

“…Throughout this section, following [DS07], we assume that there exists a positive vector θ ∈ (0, +∞) D such that, for any 1 ≤ i, j ≤ D,…”

On the use of Markovian stick-breaking priors

“…We remark that Corollary 5 is an improvement of a corresponding formula in [11] found, by different means, when X is finite and G has no nonzero entries. These formulas will be of help to identify the posterior distribution, if the Markovian stickbreaking measure is used as a prior.…”

“…The following is an improvement of a corresponding result in [11] when G has no zero entries, by directly considering the stick-breaking form of ν.…”

“…Although it was shown in [10], [11] that such Markovian stick-breaking processes connect to the limiting empirical distribution of certain simulated annealing chains, it is natural to consider their use as priors in statistical problems, the aim of this article. We first give a formula for the moments of the Markovian stick-breaking process in Theorem 4.…”