General notions of indexability for queueing control and asset management

Abstract

We develop appropriately generalized notions of indexability for problems of dynamic resource allocation where the resource concerned may be assigned more flexibility than is allowed, for example, in classical multi-armed bandits. Most especially we have in mind the allocation of a divisible resource (manpower, money, equipment) to a collection of objects (projects) requiring it in cases where its over-concentration would usually be far from optimal. The resulting project indices are functions of both a resource level and a state. They have a simple interpretation as a fair charge for increasing the resource available to the project from the specified resource level when in the specified state. We illustrate ideas by reference to two model classes which are of independent interest. In the first, a pool of servers is assigned dynamically to a collection of service teams, each of which mans a service station. We demonstrate indexability under a natural assumption that the service rate delivered is increasing and concave in the team size. The second model class is a generalization of the spinning plates model for the optimal deployment of a divisible investment resource to a collection of reward generating assets. Asset indexability is established under appropriately drawn laws of diminishing returns for resource deployment. For both model classes numerical studies provide evidence that the proposed greedy index heuristic performs strongly. This is a class of models concerned with the sequential allocation of effort, to be thought of as a single indivisible resource, to a collection of stochastic reward generating projects (or bandits as they are sometimes called). Gittins demonstrated that optimal project choices are those of highest index. There is no doubt that the idea that strongly performing policies are determined by simple, interpretable calibrations (i.e., indices) of decision options is an attractive and powerful one and offers crucial computational benefits. There is now substantial literature describing extensions to and reformulations of Gittins' result. Some key contributions are cited in the recent survey of Mahajan and Teneketzis [14].Whittle [21] introduced a class of restless bandit problems (RBPs) as a means of addressing a critical limitation of Gittins' MABs, namely, that projects should remain frozen while not in receipt of effort. In RBPs, projects may change state while active or passive though according to different dynamics. However, this generalization is bought at great cost. In contrast to MABs, RBPs are almost certainly intractable having been shown to be PSPACE-hard by Papadimitriou and Tsitsiklis [16]. Whittle [21] proposed an index heuristic for those RBPs which pass an indexability test. This heuristic reduces to Gittins' index policy in the MAB case. Whittle's index emerges from a Lagrangian relaxation of the original problem and has an interpretation as a fair charge for the allocation of effort to a particular project in a particular state. Weber and Weiss [20] established a fo...

Dynamic resource allocation in a multi-product make-to-stock production system

“…We have found that our analyses are more easily accomplished via optimisation problem (6) than via direct use of the Lagrangian relaxation (4). The use of c in place of W is well suited for proofs and calculations but, as we shall see, the equivalent problems formulated with variable W lend themselves much more directly to economic interpretation and heuristic understanding.…”

“…In order to construct strongly performing heuristics, we follow Whittle [18] and Glazebrook et al [4] in developing a Lagrangian relaxation of the original problem in which a charge is levied for the deployment of the key resource. In this relaxation the Lagrange multiplier (W ) has an economic interpretation as a charge for one unit of resource applied to production for one unit of time.…”

“…The class of heuristic policies developed in the paper can be viewed as a generalisation of the restless bandit proposal of Veatch and Wein [15] to our divisible resource model, by the deployment and development of the generalised index ideas of Glazebrook et al [4]. The work additionally develops that of Veatch and Wein by allowing for backordering of demand and by the illumination of the connection between resource idling and the failure of indexability in our more general context.…”

Robust decomposable Markov decision processes motivated by allocating school budgets

“…The authors propose heuristic techniques for dealing with several classes of constraints that use the solutions for individual MDPs to construct an approximate global solution. Glazebrook et al (2011) consider the allocation of a divisible resource, such as manpower, money, or equipment, to a collection of tasks requiring it. The authors develop a notion of indexability for problems of dynamic resource allocation where the resource concerned may be assigned more flexibility than is allowed, for example, in classical multi-armed bandits.…”

Parameter Dependent Optimal Thresholds, Indifference Levels and Inverse Optimal Stopping Problems

“…Most recently, Glazebrook et al [15] have generalised the notion of indexability to a large class of resource allocation problems. We note, however, that the definition of indexability presented here does not arise out of a Lagrangian relaxation of an original problem involving only a running reward as is the case in [15]. Instead, in the context of a optimal stopping problems, indexability is a natural feature in the monotone comparative statics of parametrised families of forward problems.…”