No abstract

No abstract

In 1988 Whittle introduced an important but intractable class of restless bandit problems which generalise the multiarmed bandit problems of Gittins by allowing state evolution for passive projects. Whittle's account deployed a Lagrangian relaxation of the optimisation problem to develop an index heuristic. Despite a developing body of evidence (both theoretical and empirical) which underscores the strong performance of Whittle's index policy, a continuing challenge to implementation is the need to establish that the competing projects all pass an indexability test. In this paper we employ Gittins' index theory to establish the indexability of (inter alia) general families of restless bandits which arise in problems of machine maintenance and stochastic scheduling problems with switching penalties. We also give formulae for the resulting Whittle indices. Numerical investigations testify to the outstandingly strong performance of the index heuristics concerned.

No abstract

Service is often provided in contexts where tasks or customers are impatient or perishable in that they have natural lifetimes of availability for useful service. Moreover, these lifetimes are usually unknown to the service provider. The question of how service might best be allocated to the currently waiting tasks or customers in such a context has been neglected and we propose three simple models. For each model, an index heuristic is developed and is assessed numerically. In all cases studied the heuristic comes close to optimality.

We propose a general Markovian model for the optimal control of admissions and subsequent routing of customers for service provided by a collection of heterogeneous stations. Queue-length information is available to inform all decisions. Admitted customers will abandon the system if required to wait too long for service. The optimisation goal is the maximisation of reward rate earned from service completions, net of the penalties paid whenever admission is denied, and the costs incurred upon every customer loss through impatience. We show that the system is indexable under mild conditions on model parameters and give an explicit construction of an index policy for admission control and routing founded on a proposal of Whittle for restless bandits. We are able to gain insights regarding the strength of performance of the index policy from the nature of solutions to the Lagrangian relaxation used to develop the indices. These insights are strengthened by the development of performance bounds. Although we are able to assert the optimality of the index heuristic in a range of asymptotic regimes, the performance bounds are also able to identify instances where its performance is relatively weak. Numerical studies are used to illustrate and support the theoretical analyses.

In this paper we consider the problem of minimizing the costs of outsourcing warranty repairs when failed items are dynamically routed to one of several service vendors. In our model, the manufacturer incurs a repair cost each time an item needs repair and also incurs a goodwill cost while an item is awaiting and undergoing repair. For a large manufacturer with annual warranty costs in the tens of millions of dollars, even a small relative cost reduction from the use of dynamic (rather than static) allocation may be practically significant. However, due to the size of the state space, the resulting dynamic programming problem is not exactly solvable in practice. Furthermore, standard routing heuristics, such as join-theshortest-queue, are simply not good enough to identify potential cost savings of any significance. We use two different approaches to develop effective, simply structured index policies for the dynamic allocation problem. The first uses dynamic programming policy improvement while the second deploys Whittle's proposal for restless bandits. The closed form indices concerned are new and the policies sufficiently close to optimal to provide cost savings over static allocation. All results of this paper are demonstrated using a simulation study.

No abstract