A survey of Markov decision models for control of networks of queues

Stidham, Shaler; Weber, Richard

doi:10.1007/bf01158935

Cited by 181 publications

(95 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The answer is not always simple or obvious, and is based on optimality criteria and the structure of the underlying system. There exists an extensive literature on dispatching policies and their optimality [9,26,27,35,36,37,42,46]. Among the dispatching policies, the join-the-shortest-queue (JSQ) policy has received considerable attention [5,6,10,19,20,23,24,28,30,44,45].…”

Section: Introductionmentioning

confidence: 99%

The shorter queue polling model

et al. 2013

View full text Add to dashboard Cite

We consider a two-queue polling model in which customers upon arrival join the shorter of two queues. Customers arrive according to a Poisson process and the service times in both queues are independent and identically distributed random variables having the exponential distribution. The two-dimensional process of the numbers of customers at the queue where the server is and at the other queue is a two-dimensional Markov process. We derive its equilibrium distribution using two methodologies: the compensation approach and a reduction to a boundary value problem.

show abstract

Section: Introductionmentioning

confidence: 99%

The shorter queue polling model

et al. 2013

View full text Add to dashboard Cite

show abstract

“…However, there are only a few problems which can be solved explicitly (cf. Sennott (1998), Kitaev/Rykov (1995), Stidham/Weber (1993)). Another possibility is to solve the optimality equation numerically by policy iteration.…”

Section: Introductionmentioning

confidence: 99%

Optimal control of queueing networks: an approach via fluid models

Bäuerle

2002

Adv. Appl. Probab.

View full text Add to dashboard Cite

We consider a general control problem for networks which includes the special cases of scheduling in multiclass queueing networks and routing problems. The fluid approximation of the network is used to derive new results about the optimal control for the stochastic network. The main emphasis lies on the average cost criterion, however the β-discounted as well as the finite cost problem are also investigated. One of our main results states that the fluid problem provides a lower bound to the stochastic network problem. For scheduling problems in multiclass queueing networks we show the existence of an average cost optimal decision rule, if the usual traffic conditions are satisfied. Moreover, we give under the same condition a simple stabilizing scheduling policy. Another important issue that we address is the construction of simple asymptotically optimal decision rules. Asymptotic optimality is here seen w.r.t. fluid scaling. We show that every minimizer of the optimality equation is asymptotically optimal. And what is more important for practical purposes, we outline a general way to identify fluid optimal feedback rules as asymptotically optimal ones. Last but not least for routing problems an asymptotically optimal decision rule is given explicitly, namely a so-called least-loaded-routing rule.

show abstract

“…See Rosberg et al [18], Weber and Stidham [28], [24], and Altman and Koole [1]. Our model is also a Markov decision model but it is di¤erent in that all uncontrollable transition rates in the network are state dependent, rather than …xed and independent of the system state.…”

Section: Motivation and Literature Reviewmentioning

confidence: 99%

Optimal Production Policies With Multistage Stochastic Demand Lead Times

Kim

Ahn

Righter

2009

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

We study the value of multi-stage advance demand information (MADI) in a production system in which customers place an order in advance of their actual need, and each order goes through multiple stages before it becomes due. Any order which is not immediately …lled at its due date will be backordered. The producer must decide whether to produce or not based on real-time information regarding current and future orders. We formulate the problem as a Markov decision process and analyze the impact of the demand information on the production policy and the cost. We show that the optimal production policy is a state-dependent base-stock policy, and we show that it has certain monotonicity properties. We also introduce a simple heuristic policy that is signi…cantly easier to compute and that inherits the structural properties of the optimal policy. In addition, we show that its base-stock levels bound those of the socially optimal policy. Numerical study identi…es the conditions under which MADI is most bene…cial and shows that the heuristic performs almost as well as the optimal policy when MADI is most bene…cial.

show abstract

A survey of Markov decision models for control of networks of queues

Abstract: We review models for the optimal control of networks of queues, Our main emphasis is on models based on Markov decision theory and the characterization of the structure of optimal control policies.

Cited by 181 publications

References 54 publications

The shorter queue polling model

The shorter queue polling model

Optimal control of queueing networks: an approach via fluid models

Optimal Production Policies With Multistage Stochastic Demand Lead Times

Contact Info

Product

Resources

About