On the General Utility of Discounted Markov Decision Processes

Kadota, Yoshinobu

doi:10.1016/s0969-6016(97)00029-4

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As far as we are aware, it appears that little work has been done on the Lagrange method to general utility-constrained MDPs. The method of analysis for general utitity functions is closely related to [1,2], in which discounted MDPs have been studied with general utility function and whose results are applied to characterize a constrained optimal policy.…”

Section: Introduction and Problem Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Discounted Markov decision processes with utility constraints

Kadota

Kurano

Yasuda

2006

Computers & Mathematics with Applications

View full text Add to dashboard Cite

We consider utility-constrained Markov decision processes. The expected utility of the total discounted reward is maximized subject to multiple expected utility constraints. By introducing a corresponding Lagrange function, a saddle-point theorem of the utility constrained optimization is derived. The existence of a constrained optimal policy is characterized by optimal action sets specified with a parametric utility.

show abstract

Section: Introduction and Problem Formulationmentioning

confidence: 99%

“…For the utility treatment for MDPs and constrained MDPs, refer to [1,2,[4][5][6][7] and their references.…”

Section: Introduction and Problem Formulationmentioning

confidence: 99%

Discounted Markov decision processes with utility constraints

Kadota

Kurano

Yasuda

2006

Computers & Mathematics with Applications

View full text Add to dashboard Cite

show abstract

“…Rieder [18] has treated the non-stationary and unbounded model, in which several results obtained in [8,9] have been extended and completed. Also, the general utility-treatment for stopped decision processes has been studied by Kadota et al [13,14]. In this paper, we consider the optimization problem for the stopped decision process over stopping times t constrained so that Et Y a for some ®xed a > 0 and develop mathematical programming methods to solve the problem.…”

Section: Introductionmentioning

confidence: 99%

Markov decision processes with a stopping time constraint

Horiguchi¹

2001

Mathematical Methods of OR

View full text Add to dashboard Cite

In this paper, the optimization problem for a stopped Markov decision process with ®nite states and actions is considered over stopping times t constrained so that Et Y a for some ®xed a > 0. The problem is solved through randomization of stopping times and mathematical programming formulation by occupation measures. Another representation, called Frepresentation, of randomized stopping times is given, by which the concept of Markov or stationary randomized stopping times is introduced. We treat two types of occupation measures, running and stopped, but stopped occupation measure is shown to be expressed by running one. We study the properties of the set of running occupation measures achieved by di¨erent classes of pairs of policies and randomized stopping times. Analyzing the equivalent mathematical programming problem formulated by running occupation measures corresponding with stationary policies and stationary randomized stopping times, we prove the existence of an optimal constrained pair of stationary policy and stopping time requiring randomization in at most one state.

show abstract

On the General Utility of Discounted Markov Decision Processes

Cited by 2 publications

References 0 publications

Discounted Markov decision processes with utility constraints

Discounted Markov decision processes with utility constraints

Markov decision processes with a stopping time constraint

Contact Info

Product

Resources

About