Markov and semi-Markov decision models and optimal stopping

Schäl, Manfred

doi:10.1007/978-1-4899-0574-1_4

Cited by 3 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The result in this work extends theorems in Schäl (1984Schäl ( , 1986, where optimal stationary policies were obtained for risk-neutral multichain dynamic programs via the discounted criterion, and in Cavazos-Cadena and Montes-de-Oca (2000a), where risk-sensitive unichain MDPs were analyzed. The strategy to prove Theorem 3.1 does not uses the discounted criterion, but is based on the construction of a new MDP satisfying the unichain property, so that, essentially, the main result of this note is obtained from Theorem 4.1 in the last mentioned work.…”

supporting

confidence: 56%

See 1 more Smart Citation

Stationary optimal policies in a class of multichain positive dynamic programs with finite state space and risk-sensitive criterion

Cavazos–Cadena

Montes-de-Oca

2001

Appl. Math. (Warsaw)

View full text Add to dashboard Cite

Abstract. This work concerns Markov decision processes with finite state space and compact action sets. The decision maker is supposed to have a constant-risk sensitivity coefficient, and a control policy is graded via the risk-sensitive expected total-reward criterion associated with nonnegative one-step rewards. Assuming that the optimal value function is finite, under mild continuity and compactness restrictions the following result is established: If the number of ergodic classes when a stationary policy is used to drive the system depends continuously on the policy employed, then there exists an optimal stationary policy, extending results obtained by Schäl (1984) for risk-neutral dynamic programming. We use results recently established for unichain systems, and analyze the general multichain case via a reduction to a model with the unichain property.

show abstract

supporting

confidence: 56%

“…In addition to Assumptions 2.1 and 2.2, this problem will be studied under the following requirement, which is referred to as the structural stability condition (Schweitzer (1968), Schäl (1986), Cavazos-Cadena, Feinberg and Montes-de-Oca (2000)). …”

Section: Main Resultmentioning

confidence: 99%

Stationary optimal policies in a class of multichain positive dynamic programs with finite state space and risk-sensitive criterion

Cavazos–Cadena

Montes-de-Oca

2001

Appl. Math. (Warsaw)

View full text Add to dashboard Cite

show abstract

On the chance to visit a goal set infinitely often

Schäl

1990

Optimization

View full text Add to dashboard Cite

A Note on the Existence of Optimal Policies in Total Reward Dynamic Programs with Compact Action Sets

2000

View full text Add to dashboard Cite

This work deals with Markov decision processes (MDPs) with expected total rewards, discrete state spaces, and compact action sets. Within this framework, a question on the existence of optimal stationary policies, formulated by Puterman (1994, p. 326), is considered. The paper concerns the possibility of obtaining an affirmative answer when additional assumptions are imposed on the decision model. Three conditions ensuring the existence of average optimal stationary policies in finite-state MDPs are analyzed, and it is shown that only the so-called structural continuity condition is a natural sufficient assumption under which the existence of total-reward optimal stationary policies can be guaranteed. In particular, this existence result holds for unichain MDPs with finite state space, but an example is provided to show that this general conclusion does not have an extension to the denumerable state space case.

show abstract

Markov and semi-Markov decision models and optimal stopping

Cited by 3 publications

References 9 publications

Stationary optimal policies in a class of multichain positive dynamic programs with finite state space and risk-sensitive criterion

Stationary optimal policies in a class of multichain positive dynamic programs with finite state space and risk-sensitive criterion

On the chance to visit a goal set infinitely often

A Note on the Existence of Optimal Policies in Total Reward Dynamic Programs with Compact Action Sets

Contact Info

Product

Resources

About