Large deviations for a class of nonhomogeneous Markov chains

Abstract

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P. Let {X_n} be the associated nonhomogeneous Markov chain where P_n controls movement from time n-1 to n. The main statements are a large deviation principle and bounds for additive functionals of the nonhomogeneous process under some regularity conditions. In particular, when P is reducible, three regimes that depend on the decay of certain ``connection'' P_n probabilities are identified. Roughly, if the decay is too slow, too fast or in an intermediate range, the large deviation behavior is trivial, the same as the time-homogeneous chain run with P or nontrivial and involving the decay rates. Examples of anomalous behaviors are also given when the approach P_n\to P is irregular. Results in the intermediate regime apply to geometrically fast running optimizations, and to some issues in glassy physics.Comment: Published at http://dx.doi.org/10.1214/105051604000000990 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Moderate deviations for nonhomogeneous Markov chains

“…Large deviations for homogeneous Markov processes can be found in several papers of Donsker and Varadhan [1,2,3,4]. The generalization to nonhomogeneous Markov chains was obtained by Dietz and Sethuraman [5]. For a survey of the theory of large deviations, the reader is referred to Varadhan [6] and references therein.…”

Some Limit Theorems of Delayed Averages for Countable Nonhomogeneous Markov Chains

“…Yang (see [15]) studied the Shannon-McMillan theorem for a nonhomogeneous Markov information source. Dietz and Sethuraman (see [4]) studied large deviations for a class of nonhomogeneous Markov chains. Yang (see [16]) studied the strong law of large numbers of bivariate functions for countable nonhomogeneous Markov chains under the condition lim n (1/n) n k=1 P k − P = 0 where P is periodic strongly ergodic.…”