A splitting technique for Harris recurrent Markov chains

Nummelin, Esa

doi:10.1007/bf00534764

Cited by 296 publications

(220 citation statements)

References 5 publications

Supporting

Mentioning

214

Contrasting

Unclassified

Order By: Relevance

“…Suppose first that = 1. We first describe the construction of a Markov chain (x n , n ≥ 0) with kernel K on an enlarged probability space as in Athreya and Ney [2] and Nummelin [13]. Fix a probability measure λ and a number ε > 0 for which condition (ii) above holds.…”

Section: Uniqueness Of Invariant Distribution In Harris Chainsmentioning

confidence: 99%

Stationary flows and uniqueness of invariant measures

Baccelli

Konstantopoulos

2010

Quart. Appl. Math.

View full text Add to dashboard Cite

Abstract. We consider a quadruple (Ω, A , ϑ, μ), where A is a σ-algebra of subsets of Ω, and ϑ is a measurable bijection from Ω into itself that preserves a finite measure μ. For each B ∈ A , we define and study the measure μ B obtained by integrating on B the number of visits to a set of the trajectory of a point of Ω before returning to B. In particular, we obtain a generalization of Kac's formula and discuss its relation to discretetime Palm theory. Although classical in appearance, its use in obtaining uniqueness of invariant measures of various stochastic models seems to be new. We apply the concept to countable Markov chains and Harris processes in general state space.

show abstract

Section: Uniqueness Of Invariant Distribution In Harris Chainsmentioning

confidence: 99%

Stationary flows and uniqueness of invariant measures

Baccelli

Konstantopoulos

2010

Quart. Appl. Math.

View full text Add to dashboard Cite

show abstract

“…In this article we investigate the same problem with the average payoff criterion. In [3] the authors study POMDP under the average cost criteria using the approach based on Athreya-Ney-Nummelin construction of pseudo-atoms ( [1], [8]) as described in [7]. In this article we extend those ideas to the zero-sum game case.…”

Section: Introductionmentioning

confidence: 99%

Zero-Sum Stochastic Games with Partial Information and Average Payoff

Saha

2013

J Optim Theory Appl

View full text Add to dashboard Cite

show abstract

“…The Q t s and p t s, which are formally defined in Section 2, are associated with the hitting times on an accessible atom introduced via the splitting construction of Athreya and Ney (1978) and Nummelin (1978).…”

Section: Introductionmentioning

confidence: 99%

“…The minorization allows for the fundamental splitting construction of Nummelin (1978Nummelin ( , 1984. Specifically, we can use (2) to write P (x, ·) as a two-component mixture…”

Section: Introductionmentioning

confidence: 99%

Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability Distribution

2006

View full text Add to dashboard Cite

Let π denote an intractable probability distribution that we would like to explore. Suppose that we have a positive recurrent, irreducible Markov chain that satisfies a minorization condition and has π as its invariant measure. We provide a method of using simulations from the Markov chain to construct a statistical estimate of π from which it is straightforward to sample. We show that this estimate is "strongly consistent" in the sense that the total variation distance between the estimate and π converges to 0 almost surely as the number of simulations grows. Moreover, we use some recently developed asymptotic results to provide guidance as to how much simulation is necessary. Draws from the estimate can be used to approximate features of π or as intelligent starting values for the original Markov chain. We illustrate our methods with two examples.2

show abstract

A splitting technique for Harris recurrent Markov chains

Cited by 296 publications

References 5 publications

Stationary flows and uniqueness of invariant measures

Stationary flows and uniqueness of invariant measures

Zero-Sum Stochastic Games with Partial Information and Average Payoff

Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability Distribution

Contact Info

Product

Resources

About