Continuous-Time Markov Chains

Anderson, William J.

doi:10.1007/978-1-4612-3038-0

Cited by 588 publications

(410 citation statements)

References 0 publications

Supporting

Mentioning

406

Contrasting

Order By: Relevance

“…It is well known (see e.g. Anderson, 1991) that almost every sample path of r(t) is a right-continuous step function. Consider a stochastic di erential delay equation with Markovian switching of the form d x(t) = f(x(t); x(t − ); t; r(t)) dt + g(x(t); x(t − ); t; r(t)) dB(t) (2.1)…”

Section: Asymptotic Stabilitymentioning

confidence: 99%

Robust stability and controllability of stochastic differential delay equations with Markovian switching

2004

View full text Add to dashboard Cite

Section: Asymptotic Stabilitymentioning

confidence: 99%

Robust stability and controllability of stochastic differential delay equations with Markovian switching

2004

View full text Add to dashboard Cite

“…The minimum Q-process can be directly constructed from the given transition rates q(· | x, π 1 t , π 2 t ); see (3.3), below. As is well known [1], [5], however, such a Q-process might not be regular, that is, we might have p min (s, x, t, S; π 1 , π 2 ) < 1 for some x ∈ S and t ≥ s ≥ 0. To ensure the regularity of a Q-process we propose the following 'drift' conditions.…”

Section: Strategiesmentioning

confidence: 98%

Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

Guo

Hernández–Lerma

2007

Adv. Appl. Probab.

View full text Add to dashboard Cite

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. We give conditions on the game's primitive data under which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using the extended infinitesimal operator associated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide a recursive way of computing (or at least approximating) the value of the game. Moreover, we present a 'martingale characterization' of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

show abstract

“…It is well known (see e.g. [2]) that ρ i − t has the exponential distribution with parameter −γ ii . Hence…”

Section: Lemmasmentioning

confidence: 99%

Stabilisation of hybrid stochastic differential equations by feedback control based on discrete‐time observations of state and mode

Song

Zheng

Luo

et al. 2017

IET Control Theory & Applications

View full text Add to dashboard Cite

This version is available at https://strathprints.strath.ac.uk/58494/ Strathprints is designed to allow users to access the research output of the University of Strathclyde. Unless otherwise explicitly stated on the manuscript, Copyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Please check the manuscript for details of any other licences that may have been applied. You may not engage in further distribution of the material for any profitmaking activities or any commercial gain. You may freely distribute both the url (https://strathprints.strath.ac.uk/) and the content of this paper for research or private study, educational, or not-for-profit purposes without prior permission or charge.Any correspondence concerning this service should be sent to the Strathprints administrator: strathprints@strath.ac.ukThe Strathprints institutional repository (https://strathprints.strath.ac.uk) is a digital archive of University of Strathclyde research outputs. It has been developed to disseminate open access research outputs, expose data about those outputs, and enable the management and persistent access to Strathclyde's intellectual output. (SDEs) by the feedback controls based on the discrete-time observations of the state. However, the feedback controls still depend on the continuous-time observations of the mode. Of course this is perfectly fine if the mode of the system is obvious (i.e. fully observable at no cost). However, it could often be the case where the mode is not obvious and it costs to identify the current mode of the system. To reduce the control cost, it is reasonable we identify the mode at the discrete times when we make observations for the state. Hence the feedback control should be designed based on the discrete-time observations of both state and mode. The aim of this paper is to show how to design such a feedback control to stabilise a given hybrid SDE. Stabilisation of Hybrid Stochastic Differential Equations by Feedback Control based on Discrete-time Observations of State and Mode

show abstract

Continuous-Time Markov Chains

Cited by 588 publications

References 0 publications

Robust stability and controllability of stochastic differential delay equations with Markovian switching

Robust stability and controllability of stochastic differential delay equations with Markovian switching

Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

Stabilisation of hybrid stochastic differential equations by feedback control based on discrete‐time observations of state and mode

Contact Info

Product

Resources

About