An index policy for dynamic pricing in cloud computing under price commitments

Borkar, Vivek S.; Ravikumar, K.; Saboo, Krishnakant

doi:10.4064/am2313-6-2017

Cited by 13 publications

(12 citation statements)

References 11 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In more complex models in which the Whittle index cannot be evaluated in closed form, the most widespread approach, which has its roots in the calibration method for the Gittins index in [49], is to apply an iterative procedure for approximately computing the index. This is done, for example, in [7][8][9][10][11][12]. Besides the drawback that the resulting index is only an approximation to the true Whittle index, this approach is typically computationally expensive.…”

Section: Review Of Related Literaturementioning

confidence: 99%

“…Besides its intrinsic interest for solving the aforementioned single-project parametric problem collection P, Whittle proposed in [2] to use the index λ * i as the basis of a widely popular heuristic for the multi-armed restless bandit problem (MARBP), in which M out of N > M restless bandit projects must be selected to be engaged at each time to maximize the value (under a discounted or long-run average criterion) earned from the N projects over an infinite horizon. For a sample of recent applications, see, for example, [5,[7][8][9][10][11][12][13][14][15][16][17][18]. While the MARBP is computationally intractable (PSPACE-hard; see [19]), the Whittle index policy is an intuitively appealing heuristic where, at each time, M projects with the highest current indices are engaged, so the Whittle index plays the role of a priority index for a project to be engaged.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Fast-Pivoting Algorithm for Whittle’s Restless Bandit Index

Niño‐Mora

2020

Mathematics

View full text Add to dashboard Cite

The Whittle index for restless bandits (two-action semi-Markov decision processes) provides an intuitively appealing optimal policy for controlling a single generic project that can be active (engaged) or passive (rested) at each decision epoch, and which can change state while passive. It further provides a practical heuristic priority-index policy for the computationally intractable multi-armed restless bandit problem, which has been widely applied over the last three decades in multifarious settings, yet mostly restricted to project models with a one-dimensional state. This is due in part to the difficulty of establishing indexability (existence of the index) and of computing the index for projects with large state spaces. This paper draws on the author’s prior results on sufficient indexability conditions and an adaptive-greedy algorithmic scheme for restless bandits to obtain a new fast-pivoting algorithm that computes the n Whittle index values of an n-state restless bandit by performing, after an initialization stage, n steps that entail (2/3)n3+O(n2) arithmetic operations. This algorithm also draws on the parametric simplex method, and is based on elucidating the pattern of parametric simplex tableaux, which allows to exploit special structure to substantially simplify and reduce the complexity of simplex pivoting steps. A numerical study demonstrates substantial runtime speed-ups versus alternative algorithms.

show abstract

Section: Review Of Related Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Fast-Pivoting Algorithm for Whittle’s Restless Bandit Index

Niño‐Mora

2020

Mathematics

View full text Add to dashboard Cite

show abstract

“…Thus, while the MARBP is generally intractable, as it is known to be PSPACE-hard (see [35]), Whittle introduced, in [29], a widely applied heuristic index policy. For a sample of recent applications, see, for example [36][37][38][39][40][41][42][43][44][45][46][47][48]. Yet, the Whittle index is only defined for a limited class of restless bandits, called indexable, and it is nontrivial to verify whether such an indexability property holds for a given model.…”

Section: Approach Via Restless Bandit Reformulation Whittle Index Amentioning

confidence: 99%

Fast Two-Stage Computation of an Index Policy for Multi-Armed Bandits with Setup Delays

Niño‐Mora

2020

Mathematics

View full text Add to dashboard Cite

We consider the multi-armed bandit problem with penalties for switching that include setup delays and costs, extending the former results of the author for the special case with no switching delays. A priority index for projects with setup delays that characterizes, in part, optimal policies was introduced by Asawa and Teneketzis in 1996, yet without giving a means of computing it. We present a fast two-stage index computing method, which computes the continuation index (which applies when the project has been set up) in a first stage and certain extra quantities with cubic (arithmetic-operation) complexity in the number of project states and then computes the switching index (which applies when the project is not set up), in a second stage, with quadratic complexity. The approach is based on new methodological advances on restless bandit indexation, which are introduced and deployed herein, being motivated by the limitations of previous results, exploiting the fact that the aforementioned index is the Whittle index of the project in its restless reformulation. A numerical study demonstrates substantial runtime speed-ups of the new two-stage index algorithm versus a general one-stage Whittle index algorithm. The study further gives evidence that, in a multi-project setting, the index policy is consistently nearly optimal.

show abstract

“…Proof. Consider the dynamic programming equation for the n-step finite horizon α discounted problem (12) with V 0 (x) = Cx, x ≥ 0. Here x →p 1 (·|x) is the conditional distribution of departures given the current state, andp 2 is the arrival distribution.…”

Section: Structural Properties Of the Value Functionmentioning

confidence: 99%

“…The passage from (15) to (16) is given in [12]. The proof has been reproduced below for sake of completeness.…”

mentioning

confidence: 99%