We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/n ζ where G is a "generator" matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = − k =i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit some different, perhaps unexpected, occupation behaviors depending on parameters.Although it is shown, on the one hand, that the position at time n converges to a point-mixture for all ζ > 0, on the other hand, the average occupation vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, is seen to converge to a constant, a pointmixture, or a distribution µ G with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of "spreading" between the cases 0 < ζ < 1 and ζ > 1.In particular, when G is appropriately chosen, intriguingly, µ G is a Dirichlet distribution, reminiscent of results in Pólya urns.
We consider the connections among 'clumped' residual allocation models (RAMs), a general class of stick-breaking processes including Dirichlet processes, and the occupation laws of certain discrete space timeinhomogeneous Markov chains related to simulated annealing and other applications. An intermediate structure is introduced in a given RAM, where proportions between successive indices in a list are added or clumped together to form another RAM. In particular, when the initial RAM is a Griffiths-Engen-McCloskey (GEM) sequence and the indices are given by the random times that an auxiliary Markov chain jumps away from its current state, the joint law of the intermediate RAM and the locations visited in the sojourns is given in terms of a 'disordered' GEM sequence, and an induced Markov chain. Through this joint law, we identify a large class of 'stick breaking' processes as the limits of empirical occupation measures for associated time-inhomogeneous Markov chains.1.1. GEM and Dirichlet measures. Consider the infinite-dimensional simplex ∆ ∞ of all all discrete (probability) distributions on N = {1, 2, . . .}. A residual
Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P. Let {X_n} be the associated nonhomogeneous Markov chain where P_n controls movement from time n-1 to n. The main statements are a large deviation principle and bounds for additive functionals of the nonhomogeneous process under some regularity conditions. In particular, when P is reducible, three regimes that depend on the decay of certain ``connection'' P_n probabilities are identified. Roughly, if the decay is too slow, too fast or in an intermediate range, the large deviation behavior is trivial, the same as the time-homogeneous chain run with P or nontrivial and involving the decay rates. Examples of anomalous behaviors are also given when the approach P_n\to P is irregular. Results in the intermediate regime apply to geometrically fast running optimizations, and to some issues in glassy physics.Comment: Published at http://dx.doi.org/10.1214/105051604000000990 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.