A New Approach to the Limit Theory of Recurrent Markov Chains

Abstract

Abstract. Let {X"; n > 0} be a Harris-recurrent Markov chain on a general state space. It is shown that there is a sequence of random times {N¡; i > 1} such that {XN.; i > 1} are independent and identically distributed. This idea is used to show that {Xn} is equivalent to a process having a recurrence point, and to develop a regenerative scheme which leads to simple proofs of the ergodic theorem, existence and uniqueness of stationary measures.

Stationary flows and uniqueness of invariant measures

“…Note that this theorem, without the positivity condition, is proved by Harris [6], Orey [14] and in the paper of Athreya and Ney [2]; the latter authors introduced the idea of splitting to provide a construction of the Harris chain on an enlarged probability space. This was also done, independently, by Nummelin [13].…”

“…Suppose first that = 1. We first describe the construction of a Markov chain (x n , n ≥ 0) with kernel K on an enlarged probability space as in Athreya and Ney [2] and Nummelin [13]. Fix a probability measure λ and a number ε > 0 for which condition (ii) above holds.…”

“…Bernoulli random variables with common law p(·); the two sequences are not independent under P x . The initial idea for this construction comes from the paper of of Athreya and Ney [2]. Here we follow the formulation and construction of Thórisson [21,Chapter 10, who shows that the times at which the pair process (x n , ζ n ) visits the set…”

Limit theorems for one-dimensional transient random walks in Markov environments*1

“…[5,6,19]), some properties of the Markov chain (x n ) that follow from Assumption 1.5. The main objectives here are to introduce the regeneration times N k and to obtain the Perron-Frobenius type Lemmas 2.6 and 2.8.…”

Zero-Sum Stochastic Games with Partial Information and Average Payoff

“…In this article we investigate the same problem with the average payoff criterion. In [3] the authors study POMDP under the average cost criteria using the approach based on Athreya-Ney-Nummelin construction of pseudo-atoms ( [1], [8]) as described in [7]. In this article we extend those ideas to the zero-sum game case.…”