Discrete-Time Controlled Markov Processes With Average Cost Criterion: A Survey

Arapostathis, Ari; Borkar, Vivek S.; Fernández-Gaucherand, E.; Ghosh, M. K.; Marcus, Solomon

doi:10.21236/ada454852

Cited by 83 publications

(144 citation statements)

References 0 publications

Supporting

Mentioning

142

Contrasting

Unclassified

Order By: Relevance

“…The average cost criterion is mathematically more difficult to analyze than the discounted cost one, especially when the state and/or action spaces are Borel and the perstage cost function is unbounded [1], which occurs in our case. We take advantage of some recurrent properties of the process and pose it as an essentially bounded cost per-stage problem, which makes results from [2,4,5] applicable.…”

Section: Introductionmentioning

confidence: 95%

Optimal communication logics in networked control systems

Hespanha

2004

2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601)

View full text Add to dashboard Cite

Abstract-This paper addresses the control of spatially distributed processes over a network that imposes bandwidth constraints and communication delays. Optimal communication policies are derived for an estimator-based Networked Control System architecture to reduce the communication load. These policies arise as solutions of an average cost optimization problem, which is solved using dynamic programming. The optimal policies are shown to be deterministic.

show abstract

Section: Introductionmentioning

confidence: 95%

Optimal communication logics in networked control systems

Hespanha

2004

2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601)

View full text Add to dashboard Cite

show abstract

“…The control problem associated to (1) when T → ∞ is an important paradigm for many applications, see for instance [7] or [8]. We also study how those bounds can be extended to the conditional second moments of the system.…”

Section: Introductionmentioning

confidence: 99%

“…These are hard tasks as the straightforward limit may not exist or may be nonunique, as for the general case one seeks the limit supremum or infimum of (1) e.g., see [7,8] in the context of Markov decision processes. The aforementioned bounds allow us to demonstrate that the long-run average cost is well defined and is finite under mean square stability.…”

Section: Introductionmentioning

confidence: 99%

Quadratic costs and second moments of jump linear systems with general Markov chain

Costa

Vargas

Val

2011

Math. Control Signals Syst.

View full text Add to dashboard Cite

This paper presents an analytic, systematic approach to handle quadratic functionals associated with Markov jump linear systems with general jumping state. The Markov chain is finite state, but otherwise general, possibly reducible and periodic. We study how the second moment dynamics are affected by the additive noise and the asymptotic behaviour, either oscillatory or invariant, of the Markov chain. The paper comprises a series of evaluations that lead to a tight two-sided bound for quadratic cost functionals. A tight two-sided bound for the norm of the second moment of the system is also obtained. These bounds allow us to show that the long-run average cost is well defined for system that are stable in the mean square sense, in spite of the periodic behaviour of the chain and taking into consideration that it may not be unique, as it may depend on the initial distribution. We also address the important question of approximation of the long-run average cost via adherence of finite horizon costs.

show abstract

“…Thus, there exist extensive bibliographies to deal with the ASPC criterion; for instance, see [14] for finite MDPs, [1,20] for denumerable MDPs and [12,13,22] for Borel MDPs. Unfortunately, all of those work above are concerned with the case of discrete-time MDPs.…”

Section: Introductionmentioning

confidence: 99%

Average sample-path optimality for continuous-time Markov decision processes in Polish spaces

Zhu

2011

Acta Math. Appl. Sin. Engl. Ser.

View full text Add to dashboard Cite

In this paper we study the average sample-path cost (ASPC) problem for continuous-time Markov decision processes in Polish spaces. To the best of our knowledge, this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces. The corresponding transition rates are allowed to be unbounded, and the cost rates may have neither upper nor lower bounds. Under some mild hypotheses, we prove the existence of ε (ε ≥ 0)-ASPC optimal stationary policies based on two different approaches: one is the "optimality equation" approach and the other is the "two optimality inequalities" approach.

show abstract

Discrete-Time Controlled Markov Processes With Average Cost Criterion: A Survey

Cited by 83 publications

References 0 publications

Optimal communication logics in networked control systems

Optimal communication logics in networked control systems

Quadratic costs and second moments of jump linear systems with general Markov chain

Average sample-path optimality for continuous-time Markov decision processes in Polish spaces

Contact Info

Product

Resources

About