Topics in the Constructive Theory of Countable Markov Chains

Abstract

Markov chains are an important idea, related to random walks, which crops up widely in applied stochastic analysis. They are used, for example, in performance modelling and evaluation of computer networks, queuing networks, and telecommunication systems. The main point of the present book is to provide methods, based on the construction of Lyapunov functions, of determining when a Markov chain is ergodic, null recurrent, or transient. These methods can also be extended to the study of questions of stability. Of particular concern are reflected random walks and reflected Brownian motion. The authors provide not only a self-contained introduction to the theory but also details of how the required Lyapunov functions are constructed in various situations.

Fcfs infinite bipartite matching of servers and customers

“…The first and last quadrants in turn have a reflecting boundary with quadrants involving the next members of the incompatibility chain. Fayolle et al [10] also characterized the ergodicity and transience of homogeneous random walks on complexes of two-dimensional quadrants. However, by assuming that all jumps downwards are of size no more than one in each direction, they did not allow for reflections and crossovers between neighboring quadrants.…”

“…We use the same type of Lyapunov function which was developed in the monograph of Fayolle et al [10] for the homogeneous random walk on Z 2 + . In fact, our proof is very close to the one given in [10,Section 3.3].…”

“…We use the same type of Lyapunov function which was developed in the monograph of Fayolle et al [10] for the homogeneous random walk on Z 2 + . In fact, our proof is very close to the one given in [10,Section 3.3]. The random walks considered by Fayolle et al have downward jumps that do not exceed 1 in any direction, so they have no 'reflection' at the axes, which makes them more homogeneous.…”

Stability and performance analysis of networks supporting services with rate control-could the Internet be unstable?

“…We will show positive recurrence by constructing a Lyapunov function [23], [10]. For our system, a Lyapunov function is any function V: 7/~ --+ R+ such that there exists a finite set K C_ Z+ ~, where sup QV(~) < 0 (12) nf[K with QV as defined in (11).…”

Multi-Particle Processes with Reinforcements

“…The proof of the above lemma can be thought of as an exercise on use of harmonic functions in stochastic processes. It can be done with Lyapunov functions (see [6]), or alternatively with conductivities.…”