Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems

Bubeck, Sébastien; Cesa-Bianchi, Nicolò

doi:10.1561/9781601986276

Cited by 1,082 publications

(1,364 citation statements)

References 85 publications

Supporting

Mentioning

1,345

Contrasting

Unclassified

Order By: Relevance

“…An enormous body of work has accumulated over the years. While various subsets of this work have been covered in depth in several books and surveys (Berry and Fristedt, 1985;Cesa-Bianchi and Lugosi, 2006;Gittins et al, 2011;Bubeck and Cesa-Bianchi, 2012), this book provides a more textbook-like treatment of the subject.…”

Section: Prefacementioning

confidence: 99%

Introduction to Multi-Armed Bandits

Slivkins

2019

FNT in Machine Learning

334

View full text Add to dashboard Cite

show abstract

Section: Prefacementioning

confidence: 99%

Introduction to Multi-Armed Bandits

Slivkins

2019

FNT in Machine Learning

334

View full text Add to dashboard Cite

show abstract

“…At the th time slot, the genetic algorithm will be used to obtain the minimum value of 2 when each mobile device's battery energy level is sufficient for local execution or offloaded execution. Otherwise, the key-value pair method will be used by virtue of -greedy policy [17] .…”

Section: ∑̃mentioning

confidence: 99%

“…if , = 1 then 15: obtain the th mobile device and the corresponding th MEC server, then insert them into the map with key and value . 16: end if 17 Obtain the key-value pair " -" with the minimum server ( ).…”

mentioning

confidence: 99%

See 1 more Smart Citation

QoE Aware and Cell Capacity Enhanced Computation Offloading for Multi-Server Mobile Edge Computing Systems with Energy Harvesting Devices

Zhao¹,

Du²,

Liu³

et al. 2019

EasyChair Preprints

View full text Add to dashboard Cite

The increasing complexity of intelligent services requires new paradigm to overcome the problems caused by resource-limited mobile devices. Mobile edge computing systems with energy harvesting devices is such a promising technology. By offloading the computation tasks from the mobile devices to the MEC servers, users could experience services with low latency. In addition, energy harvesting technology releases the tension between high energy consumption of intelligent services and capacity-constrained mobile device batteries. However, in multi-user and multi-server scenarios where mobile devices can move arbitrarily, computation offloading strategies are faced with new challenges because of resource competition and server selection. In this paper, we will develop an intelligent computation offloading strategy. The quality of user experience cost and the cell capacity in terms of the ratio of computation tasks offloaded will be adopted as the performance metrics. An online algorithm, namely, the LODCO-Based Genetic Algorithm with Greedy Policy, will be proposed. Specifically, the algorithm is based on Lyapunov Optimization and the LODCO Algorithm. By choosing the execution mode among local execution, offloading execution and task dropping for each mobile device, our algorithm can asymptotically obtain the optimal results for the whole system. The algorithm proposed is low-complexity and could work without too much priori knowledge. Moreover, the algorithm not only inherits every advantage from the LODCO Algorithm but also adapts perfectly to the more complex environment. Simulation results illustrate that the algorithms could improve the ratio of offloading computation tasks by more than 10% while the QoE is guaranteed.

show abstract

“…In the context of learning bandits, we like to mention the work [36], where optimal control in a changing environment is studied. Here, the laws according to which the bandits evolve and receive reward/cost, are not known to the decision maker [14]. In [36] it is assumed that the (unknown) laws depend on the modulated environment.…”

Section: Related Workmentioning

confidence: 99%

Asymptotic Optimal Control of Markov-Modulated Restless Bandits

Durán

Verloop

2019

SIGMETRICS Perform. Eval. Rev.

View full text Add to dashboard Cite

This paper studies optimal control subject to changing conditions. This is an area that recently received a lot of attention as it arises in numerous situations in practice. Some applications being cloud computing systems where the arrival rates of new jobs uctuate over time, or the time-varying capacity as encountered in power-aware systems or wireless downlink channels. To study this, we focus on a restless bandit model, which has proved to be a powerful stochastic optimization framework to model scheduling of activities. In particular, it has been extensively applied in the context of optimal control of computing systems. This paper is a rst step to its optimal control when restless bandits are subject to changing conditions, the latter being modeled by Markov-modulated environments. We consider the restless bandit problem in an asymptotic regime, which is obtained by letting the population of bandits grow large, and letting the environment change relatively fast. We present su cient conditions for a policy to be asymptotically optimal and show that a set of priority policies satis es these. Under an indexability assumption, an averaged version of Whittle's index policy is proved to be inside this set of asymptotic optimal policies. The performance of the averaged Whittle's index policy is numerically evaluated for a multi-class scheduling problem in a wireless downlink subject to changing conditions. While keeping the number of bandits constant, we observe that the average Whittle index policy becomes close to optimal as the speed of the modulated environment increases.

show abstract

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems

Cited by 1,082 publications

References 85 publications

Introduction to Multi-Armed Bandits

Introduction to Multi-Armed Bandits

QoE Aware and Cell Capacity Enhanced Computation Offloading for Multi-Server Mobile Edge Computing Systems with Energy Harvesting Devices

Asymptotic Optimal Control of Markov-Modulated Restless Bandits

Contact Info

Product

Resources

About