Markov chains are an important idea, related to random walks, which crops up widely in applied stochastic analysis. They are used, for example, in performance modelling and evaluation of computer networks, queuing networks, and telecommunication systems. The main point of the present book is to provide methods, based on the construction of Lyapunov functions, of determining when a Markov chain is ergodic, null recurrent, or transient. These methods can also be extended to the study of questions of stability. Of particular concern are reflected random walks and reflected Brownian motion. The authors provide not only a self-contained introduction to the theory but also details of how the required Lyapunov functions are constructed in various situations.
In this paper we get some sufficient conditions for the finiteness or nonfiniteness of the passage-time moments for nonnegative discrete parameter processes. The developed criteria are closely connected with the well-known results of Foster for the ergodicity of Markov chains and are Ž . given in terms of sub super martingales. Then, as an application of the obtained results, we get explicit conditions for the finiteness or nonfiniteness of passage-time moments for reflected random walks in a quadrant with zero drift in the interior.
Stochastic systems provide powerful abstract models for a variety of important real-life applications: for example, power supply, traffic flow, data transmission. They (and the real systems they model) are often subject to phase transitions, behaving in one way when a parameter is below a certain critical value, then switching behaviour as soon as that critical value is reached. In a real system, we do not necessarily have control over all the parameter values, so it is important to know how to find critical points and to understand system behaviour near these points. This book is a modern presentation of the 'semimartingale' or 'Lyapunov function' method applied to near-critical stochastic systems, exemplified by non-homogeneous random walks. Applications treat near-critical stochastic systems and range across modern probability theory from stochastic billiards models to interacting particle systems. Spatially non-homogeneous random walks are explored in depth, as they provide prototypical near-critical systems.
We consider a one-dimensional nearest-neighbor interacting particle system, which is a mixture of the simple exclusion process and the voter model. The state space is taken to be the countable set of the configurations that have a finite number of particles to the right of the origin and a finite number of empty sites to the left of it. We obtain criteria for the ergodicity and some other properties of this system using the method of Lyapunov functions.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.