Restless bandits: activity allocation in a changing world

Whittle, Peter

doi:10.1017/s0021900200040420

Cited by 425 publications

(1,026 citation statements)

References 1 publication

Supporting

Mentioning

1,021

Contrasting

Unclassified

Order By: Relevance

“…Gittins derived this result as a by-product of his ground-breaking results on the multi-armed bandit problem. The literature of multiarmed bandit related papers that build on Gittins's result is huge, see, e.g., [3,[20][21][22][23]. However, Gittins's optimality result in the context of the M/G/1 queue has not been fully exploited, and it has not received the attention it deserves.…”

Section: Introductionmentioning

confidence: 99%

On the Gittins index in the M/G/1 queue

Aalto¹,

2009

View full text Add to dashboard Cite

For an M/G/1 queue with the objective of minimizing the mean number of jobs in the system, the Gittins index rule is known to be optimal among the set of non-anticipating policies. We develop properties of the Gittins index. For a singleclass queue it is known that when the service time distribution is of type Decreasing Hazard Rate (New Better than Used in Expectation), the Foreground-Background (First-Come-First-Served) discipline is optimal. By utilizing the Gittins index approach, we show that in fact, Foreground-Background and First-Come-First-Served are optimal if and only if the service time distribution is of type Decreasing Hazard Rate and New Better than Used in Expectation, respectively. For the multi-class case, where jobs of different classes have different service distributions, we obtain new results that characterize the optimal policy under various assumptions on the service time distributions. We also investigate distributions whose hazard rate and mean residual lifetime are not monotonic.

show abstract

Section: Introductionmentioning

confidence: 99%

On the Gittins index in the M/G/1 queue

Aalto¹,

2009

View full text Add to dashboard Cite

show abstract

“…Building upon the seminal work of Whittle (1988), Dusonchet (2003) and Niño-Mora (2007) formulate the multi-item make-to-stock queue with average cost criterion as a Restless Bandit Problem. Niño-Mora (2007) shows that the marginal productivity index obtained from the restless bandit formulation coincides with the myopic(T) scheduling policy in the case of linear holding and backordering costs, and further extends it to models with convex nonlinear cost rates and/or discounted costs.…”

Section: Literature Reviewmentioning

confidence: 99%

Dynamic control in multi-item production/inventory systems

2016

View full text Add to dashboard Cite

We consider a production/inventory system consisting of one production line and multiple products. Finished goods are kept in stock to serve stochastic demand. Demand is fulfilled immediately if there is an item of the requested product in stock and otherwise it is backordered and fulfilled later. The production line is modeled as a non-preemptive single server and the objective is to minimize the sum of the average inventory holding costs and backordering costs. We investigate the structure of the optimal production policy, propose a new scheduling policy, and develop a method for calculating base stock levels under an arbitrary but given scheduling policy. The performance of the various production policies is evaluated in extensive numerical experiments.

show abstract

“…We therefore seek heuristic policies whose performance come close to being cost minimising. An approach espoused by Whittle [15] is to seek a calibration of each machine in the form of an index, namely a real-valued function defined on its state space. Write γ m : N × R + × (R + ∪ { * }) → R + for the (continuous) index for machine m, which in the current context would be some measure of congestion at that machine moderated to take account of machine availability.…”

Section: Commentsmentioning

confidence: 99%

Dynamic routing to heterogeneous collections of unreliable servers

Glazebrook

Kirkbride

2006

Queueing Syst

View full text Add to dashboard Cite

We argue the importance of problems concerning the dynamic routing of tasks for service in environments where the servers have diverse characteristics and are subject to breakdown. We propose a general model in which both service times and repair times at each machine are i.i.d. with some general distribution. Routing decisions take account of queue lengths, machine states (up or down), the elapsed processing times of jobs in service and the times to date of any machine repairs in progress. We develop an approach to machine calibration which yields a machine index which is a function of all of the preceding information. The heuristic which routes all tasks to the machine of current smallest index performs outstandingly well. The approach of the paper is flexible and is capable of yielding strongly performing routing policies for a range of variants of the basic model. These include cases where job processing is lost at each breakdown and where the machine state may be only partially observed.

show abstract

Restless bandits: activity allocation in a changing world

Cited by 425 publications

References 1 publication

On the Gittins index in the M/G/1 queue

On the Gittins index in the M/G/1 queue

Dynamic control in multi-item production/inventory systems

Dynamic routing to heterogeneous collections of unreliable servers

Contact Info

Product

Resources

About