Markov Chains and Stochastic Stability

Abstract

Meyn and Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Persistence of structured populations in random environments

“…However there is a powerful and well developed theory of Markov chains on general state spaces (see e.g. Duflo 1997 or Meyn andTweedie 1993), and numerous fine criteria ensuring both existence and uniqueness of the invariant probability. For instance when S is finite (Examples 1 and 4) existence always holds and uniqueness follows from the irreducibility of P. Example 2 is a particular case of uniquely ergodic map on an Abelian compact group (Mañé 1983).…”

An Adaptive Algorithm for Finding the Optimal Base-Stock Policy in Lost Sales Inventory Systems with Censored Demand

“…Before we proceed, we recall the definition of ergodicity (see Chapter 13 of Meyn and Tweedie [14] for more details). The Markov chain X(S) = (X t (S), X t (S)) ∈ τ + × N τ : t ≥ 1 is ergodic if there exists a random variable (X ∞ (S), X ∞ (S)) such that for any initial state (…”

Stationary flows and uniqueness of invariant measures

“…This was also done, independently, by Nummelin [13]. [See also Meyn and Tweedie [10].] Our contribution here is that the uniqueness proof is a direct corollary of the results of Section 2, in the case of positive Harris recurrence, with an appeal to the ergodic theorem.…”

“…which can be seen to be invariant for K [see Meyn and Tweedie [10] and Asmussen [1]]. We now wish to show that π * is the unique invariant distribution for K. Let π be a probability measure on (S, S ) which is invariant for the kernel K. On the enlarged probability space Ω := S Z * of two-sided sequences ω = (ω n = (x n , ζ n ), n ∈ Z), consider the probability measure P under which (x n , ζ n ), n ∈ Z, is a stationary Markov chain with kernel K * and initial distribution π × p. Define the natural shift ϑ on Ω:…”