Controlled Semi-Markov Chains with Risk-Sensitive Average Cost Criterion

Chávez-Rodríguez, Selene; Cavazos–Cadena, Rolando; Cruz−Suárez, Hugo

doi:10.1007/s10957-016-0916-z

Cited by 10 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…x is determined on the Borel family of the space H ∞ = ∞ n=0 (K × [0, ∞)) of all possible realizations of the process {(X n , A n , S n )} [10,18] and E π x stands for the corresponding expectation operator. Remark 2.1 It was shown in Lemma 4.1 in [6] that Assumption 2.1 implies that N t has light tails, i. e., for every β ∈ (0, 1) and t > 0,…”

Section: Policies and Performance Criterionmentioning

confidence: 99%

“…In addition to play an important role to establish the uniqueness result in Lemma 2.1(iii), Assumption 2.2 is also crucial to ensure the existence of a solution of the optimality equation (7); see [4,5] or [6].…”

Section: Remark 22mentioning

confidence: 99%

“…On the other hand, the literature on semi-Markov models with risksensitive criteria is somewhat scarce. In [5] (uncontrolled) semi-Markov chains were studied and the risk-sensitive average cost was characterized via a Poisson equation, a result that was generalized to the controlled case in [6], whereas semi-Markov models with a finite decision horizon are studied in [13].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

Camilo-Garay

Cavazos–Cadena

Cruz−Suárez

2021

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

This work concerns with semi-Markov decision chains evolving on a finite state space. The controller has a positive and constant risk sensitivity coefficient, and the performance of a control policy is measured by the risk-sensitive average cost criterion. Under conditions ensuring that the optimal value function is determined via a single optimality equation, the fixed points of a family of contractive operators are used to obtain convergent approximations to the optimal average cost and to a solution of optimality equation, extending the classical discounted approach to the context of the paper. In contrast with the Markovian case, the contractive operators utilized in this work depend on two parameters.

show abstract

Section: Policies and Performance Criterionmentioning

confidence: 99%

Section: Remark 22mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

Camilo-Garay

Cavazos–Cadena

Cruz−Suárez

2021

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…Risk-sensitive infinite horizon discounted cost problem is considered in [5]. In [8], the authors consider the risk-sensitive average cost criterion for semi-Markov processes. But to the best of our knowledge, the present paper is the first work on risk-sensitive semi-Markov games.…”

Section: Introductionmentioning

confidence: 99%

Zero and Non-zero Sum Risk-sensitive Semi-Markov Games

Bhabak¹,

Saha²

2021

Preprint

View full text Add to dashboard Cite

In this article we consider zero and non-zero sum risk-sensitive average criterion games for semi-Markov processes with a finite state space. For the zero-sum case, under suitable assumptions we show that the game has a value. We also establish the existence of a stationary saddle point equilibrium. For the non-zero sum case, under suitable assumptions we establish the existence of a stationary Nash equilibrium.

show abstract

“…As is well known, risk-sensitive control is a subject of significant interest in the field of Markov decision processes (MDPs) and has received much attention; see, for example, [2], [5], [8], and [14] for average cost discrete-time MDPs (DTMDPs), [2], [7], and [13] for infinitehorizon discounted cost DTMDPs, [2] and [7] for finite-horizon discounted or undiscounted cost DTMDPs, [10] and [22] for average cost continuous-time MDPs (CTMDPs), and [10] for finite-horizon cost and infinite-horizon discounted cost CTMDPs. Regarding risk-sensitive semi-Markov decision processes (SMDPs), to the best of the authors' knowledge this issue was only addressed in [6], in which the authors concentrated on a long-run average cost. In this paper we also study risk-sensitive SMDPs, but we focus on a finite-horizon total cost.…”

Section: Introductionmentioning

confidence: 99%

Risk-sensitive semi-Markov decision processes with general utilities and multiple criteria

Huang¹,

Lian

Guo³

2018

Adv. Appl. Probab.

View full text Add to dashboard Cite

In this paper we investigate risk-sensitive semi-Markov decision processes with a Borel state space, unbounded cost rates, and general utility functions. The performance criteria are several expected utilities of the total cost in a finite horizon. Our analysis is based on a type of finite-horizon occupation measure. We express the distribution of the finite-horizon cost in terms of the occupation measure for each policy, wherein the discount is not needed. For unconstrained and constrained problems, we establish the existence and computation of optimal policies. In particular, we develop a linear program and its dual program for the constrained problem and, moreover, establish the strong duality between the two programs. Finally, we provide two special cases of our results, one of which concerns the discrete-time model, and the other the chance-constrained problem.

show abstract

Controlled Semi-Markov Chains with Risk-Sensitive Average Cost Criterion

Cited by 10 publications

References 15 publications

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

Zero and Non-zero Sum Risk-sensitive Semi-Markov Games

Risk-sensitive semi-Markov decision processes with general utilities and multiple criteria

Contact Info

Product

Resources

About