On the structure of value functions for threshold policies in queueing models

Bhulai, Sandjai; Koole, Ger

doi:10.1017/s0021900200019598

Cited by 21 publications

(33 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, the time-average costs of assignment to the queueing system are just the summed time-average costs of the two M/M/1 queues with holding costs γ 1 and γ 2 respectively (and of fixed costs γ 1 + γ 3 ). For a M/M/1 queue we know (see [6]) that g = ρ 1−ρ h, with ρ = λ/µ the system load, λ the arrival rate, µ the service rate, and holding costs h. This explains the Eq. (22).…”

Section: Substitution Into Eq (20) Yieldsmentioning

confidence: 97%

On the Control of a Queueing System with Aging State Information

2015

Self Cite

View full text Add to dashboard Cite

We investigate control of a queueing system in which a component of the state space is subject to aging. The controller can choose to forward incoming queries to the system (where it needs time for processing), or respond with a previously generated response (incurring a penalty for not providing a fresh value). Hence, the controller faces a trade-off between data freshness and response times. We model the system as a complex Markov Decision Process, simplify it, and construct a control policy. This policy shows near-optimal performance and achieves lower costs than both a myopic policy and a threshold policy.

show abstract

Section: Substitution Into Eq (20) Yieldsmentioning

confidence: 97%

On the Control of a Queueing System with Aging State Information

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…Because the resulting policy will be too complicated to repeat the policy evaluation step, the algorithm stops here. In practice, for a suitably chosen approximation, the resulting policy is nearly optimal (see, e.g., Bhulai and Koole [2], Koole and Nain [13], Ott and Krishnan [17], and Sassen et al [20]). …”

Section: Scenario 1: a Call Center With No Waiting Roommentioning

confidence: 99%

“…In G (2) s there might be more than one group to which one can route. The initial policy routes the call according to fixed probabilities to the groups in G (2) s ; that is, it splits the overflow stream from G (1) s into fixed fractions over the groups in G (2) s . The call progresses through the hierarchy whenever it is routed to a group with no idle agents until it is served at one of the groups or it is blocked at G (N) s eventually.…”

Section: Initial Policymentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Routing Policies for Multiskill Call Centers

Bhulai¹

2008

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

We consider the problem of routing calls dynamically in a multiskill call center. Calls from different skill classes are offered to the call center according to a Poisson process. The agents in the center are grouped according to their heterogeneous skill sets that determine the classes of calls they can serve. Each agent group serves calls with independent exponentially distributed service times. We consider two scenarios. The first scenario deals with a call center with no buffers in the system, so that every arriving call either has to be routed immediately or has to be blocked and is lost. The objective in the system is to minimize the average number of blocked calls. The second scenario deals with call centers consisting of only agents that have one skill and fully cross-trained agents, where calls are pooled in common queues. The objective in this system is to minimize the average number of calls in the system. We obtain nearly optimal dynamic routing policies that are scalable with the problem instance and can be computed online. The algorithm is based on one-step policy improvement using the relative value functions of simpler queuing systems. Numerical experiments demonstrate the good performance of the routing policies. Finally, we discuss how the algorithm can be used to handle more general cases with the techniques described in this article.

show abstract

“…That is, given any > 0, if we let [34] applied the rollout technique combined with neuro-dynamic programming [5] to a vehicle routing problem, Ott and Krishnan [30] and Kolarov and Hui [22] studied network routing problems, Bhulai and Koole [7] consider a multi-server queueing problem, and Koole and Nain [24] consider a two-class single-server queueing model under a preemptive priority rule. In particular, [7] and [24] obtained explicit form expressions for the value function of a fixed threshold policy, which plays the role of the heuristic base policy, and showed numerically that the rollout policy generated from the threshold policy behaves almost optimally.…”

Section: Theorem 41 Assume That Assumption 21 Holds Consider the Hmentioning

confidence: 99%

Approximate Receding Horizon Approach for Markov Decision Processes: Average Award Case

Chang¹,

Marcus²

2002

View full text Add to dashboard Cite

ISR develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical, heterogeneous and dynamic problems of engineering technology and systems for industry and government. ISR is a permanent institute of the University of Maryland, within the Glenn L. Martin Institute of Technol AbstractWe consider an approximation scheme for solving Markov Decision Processes (MDPs) with countable state space, finite action space, and bounded rewards that uses an approximate solution of a fixed finite-horizon sub-MDP of a given infinite-horizon MDP to create a stationary policy, which we call "approximate receding horizon control". We first analyze the performance of the approximate receding horizon control for infinite-horizon average reward under an ergodicity assumption, which also generalizes the result obtained by White [36]. We then study two examples of the approximate receding horizon control via lower bounds to the exact solution to the sub-MDP. The first control policy is based on a finite-horizon approximation of Howard's policy improvement of a single policy and the second policy is based on a generalization of the single policy improvement for multiple policies. Along the study, we also provide a simple alternative proof on the policy improvement for countable state space. We finally discuss practical implementations of these schemes via simulation.

show abstract

On the structure of value functions for threshold policies in queueing models

Cited by 21 publications

References 4 publications

On the Control of a Queueing System with Aging State Information

On the Control of a Queueing System with Aging State Information

Dynamic Routing Policies for Multiskill Call Centers

Approximate Receding Horizon Approach for Markov Decision Processes: Average Award Case

Contact Info

Product

Resources

About