Markov Chains in Random Environments: The Case of Markovian Environments

Cogburn, Robert

doi:10.1214/aop/1176994620

Cited by 74 publications

(27 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Optimization of service rate for the case when arrival rate is a Markov process is studied in [26]. See also, a birthdeath process in random environment [13] and a Markov chain in Markov environment, studied in [12,16,33]. A particular case of a Markov-modulated setting is when the service dynamics is subject to interruptions.…”

Section: 1mentioning

confidence: 99%

Stationary distributions and convergence for M/M/1 queues in interactive random environment

et al. 2020

View full text Add to dashboard Cite

A Markovian single-server queue is studied in an interactive random environment. The arrival and service rates of the queue depend on the environment, while the transition dynamics of the random environment depends on the queue length. We consider in detail two types of Markov random environments: a pure jump process and a reflected jump-diffusion. In both cases, the joint dynamics is constructed so that the stationary distribution can be explicitly found in a simple form (weighted geometric). We also derive an explicit estimate for exponential rate of convergence to the stationary distribution via coupling.

show abstract

Section: 1mentioning

confidence: 99%

Stationary distributions and convergence for M/M/1 queues in interactive random environment

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Repeating this step n times, we obtain Eq. (6). Applying (M1), for bounded measurable functions g and h from 1 ( ) t + Θ X andX to R respectively, we have…”

Section: The Equivalence Theoremsmentioning

confidence: 99%

“…Kalikow [2] , Sinai [3] , and Sznaitman [4] studied deeply the model. At the same time, Nawrotzki [5] , and Cogburn [6][7][8] , also established a general model of Markov chains in random environments, and Orey [9] reviewed the works on this field and also gave some new results and open problems in a special invited paper. Please notice that the model of Solomon is space random environments, i.e.…”

mentioning

confidence: 99%

Model of Markov chains in space-time random environments

Yang

2007

Wuhan Univ. J. of Nat. Sci.

View full text Add to dashboard Cite

A general framework of stochastic model for a Markov chain in a space-time random environment is introduced, here the environment * , ,X is a random field. We study the dependence relations between the environment and the original chain, especially the "feedback". Some equivalence theorems and law of large numbers are obtained.The theory of stochastic processes in random environments has been pursued for some time, especially the random walk in random environment. Solomon [1] firstly gave the model of random walks in random environments. Kalikow [2] , Sinai [3] , and Sznaitman [4] studied deeply the model. At the same time, Nawrotzki [5] , and Cogburn [6][7][8] , also established a general model of Markov chains in random environments, and Orey [9] reviewed the works on this field and also gave some new results and open problems in a special invited paper. Please notice that the model of Solomon is space random environments, i.e. the environments only depend on the spatial locations, not evolution as time, and the model of Cogburn only considers the time but not the spatial locations of the original chain. Bérard [10] and Rassoul-Agha [11] studied a class of random walk in a space-time random environment in a sense, and obtained some results, such as central limit theorems and invariance principle. In the present paper, we establish a general formulation of Markov chains in space-time random environments, and give some equivalence theorems. Finally, we obtain a kind of law of large numbers of the model for general environments.

show abstract

“…For details, see the Refs. [1][2][3][4][5][6][7][8][9][10], for Markov chains in random environment, the ergodic of recurrent, positive recurrent and the period of a state of Markov chains in random environment had been introduced. Li given the condition of m-irreducible in Ref.…”

Section: Introductionmentioning

confidence: 99%

The classification of states for Markov chain in random environments and m-irreducible

Wang

2007

Wuhan Univ. J. of Nat. Sci.

View full text Add to dashboard Cite

First of all, we introduces the concept of m-irreducible of Markov chain in random environment. Then under the condition of m-irreducible, the relationship of recurrent and positive recurrent between two states is studied. We also give several conditions that can imply a state is recurrent and positive recurrent.And then the period of a state is discussed and we obtained that under the condition of m-irreducible, for any two states in X , they have the same period.

show abstract

Markov Chains in Random Environments: The Case of Markovian Environments

Cited by 74 publications

References 0 publications

Stationary distributions and convergence for M/M/1 queues in interactive random environment

Stationary distributions and convergence for M/M/1 queues in interactive random environment

Model of Markov chains in space-time random environments

The classification of states for Markov chain in random environments and m-irreducible

Contact Info

Product

Resources

About