Augmenting Max-Weight With Explicit Learning for Wireless Scheduling With Switching Costs

Krishnasamy, Subhashini; Akhil, P T; Arapostathis, Ari; Sundaresan, Rajesh; Shakkottai, Sanjay

doi:10.1109/tnet.2018.2869874

Cited by 24 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• We provide a significant generalisation of the odd arm identification problem in [4], which dealt with the special case of Poisson observations, to the case of general vector exponential family observations. • We modify the policy in [4] to incorporate switching costs based on the idea of slowed switching in [1], [8] and [9]. • We show that the proposed policy, which incorporates learning, is asymptotically optimal even with switching costs; the growth rate of the total cost, as the probability of false detection and the switching parameter are driven to zero, is the same as that without switching costs.…”

Section: A Our Contributionsmentioning

confidence: 99%

Learning to Detect an Anomalous Target with Observations from an Exponential Family

Prabhu

Bhashyam

Gopalan

et al. 2019

2019 IEEE Data Science Workshop (DSW)

Self Cite

View full text Add to dashboard Cite

The problem of detecting an odd arm from a set of K arms of a multi-armed bandit, with fixed confidence, is studied in a sequential decision-making scenario. Each arm's signal follows a distribution from a vector exponential family. All arms have the same parameters except the odd arm. The actual parameters of the odd and non-odd arms are unknown to the decision maker. Further, the decision maker incurs a cost for switching from one arm to another. This is a sequential decision making problem where the decision maker gets only a limited view of the true state of nature at each stage, but can control his view by choosing the arm to observe at each stage. Of interest are policies that satisfy a given constraint on the probability of false detection. An information-theoretic lower bound on the total cost (expected time for a reliable decision plus total switching cost) is first identified, and a variation on a sequential policy based on the generalised likelihood ratio statistic is then studied. Thanks to the vector exponential family assumption, the signal processing in this policy at each stage turns out to be very simple, in that the associated conjugate prior enables easy updates of the posterior distribution of the model parameters. The policy, with a suitable threshold, is shown to satisfy the given constraint on the probability of false detection. Further, the proposed policy is asymptotically optimal in terms of the total cost among all policies that satisfy the constraint on the probability of false detection.

show abstract

Section: A Our Contributionsmentioning

confidence: 99%

Learning to Detect an Anomalous Target with Observations from an Exponential Family

Prabhu

Bhashyam

Gopalan

et al. 2019

2019 IEEE Data Science Workshop (DSW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The idle power is the basic consumption incurred when an ARC group is activated (which means the hardware components supporting the ARC group must be powered on). At the same time, the operational power is consumed when ARC units are engaged in processing tasks [11]. The relevant components of ARCs, deployed in FNs such as micro BSs or embedded servers, can be de-activated when not in use for power saving, and activated upon the arrival of a task occupying at least one ARC unit [14] 2 .…”

Section: Network Modelmentioning

confidence: 99%

Energy Efficient Priority-Based Task Scheduling for Computation Offloading in Fog Computing

Yin¹,

Fu²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Fog computing offers a flexible solution for computational offloading for Internet of Things (IoT) services at the edge of wireless networks. It serves as a complement to traditional cloud computing, which is not cost-efficient for most offloaded tasks in IoT applications involving small-to-medium levels of computing tasks. Given the heterogeneity of tasks and resources in fog computing, it is vital to offload each task to an appropriate destination to fully utilize the potential benefit of this promising technology. In this paper, we propose a scalable priority-based index policy, referred to as the Prioritized Incremental Energy Rate (PIER), to optimize the energy efficiency of the network. We demonstrate that PIER is asymptotically optimal in a special case applicable for local areas with high volumes of homogeneous offloaded tasks and exponentially distributed task durations. In more general cases with statistically different offloaded tasks, we further demonstrate the improvement of PIER over benchmark policies in terms of energy efficiency and the robustness of PIER to different task duration distributions by extensive simulations. Our results show that PIER can perform better than benchmark policies in more than 78.6% of all simulation runs.

show abstract

“…The novelty here is that instead of working with queues of jobs, the decision maker is trying to minimize certain information backlog that corresponds to "uncertainty" in the system. Another line of work investigates how to augment MaxWeight with explicit learning and side information to improve performance (Krishnasamy et al 2018, Neely et al 2012). However, they tend to focus more on parameter estimation, rather than approachabilitiy proporties for the algorithm.…”

Section: Literature Reviewmentioning

confidence: 99%

Learning and Information in Stochastic Networks and Queues

Walton¹,

Xu²

2021

Preprint

View full text Add to dashboard Cite

We review the role of information and learning in the stability and optimization of queueing systems. In recent years, techniques from supervised learning, bandit learning and reinforcement learning have been applied to queueing systems supported by increasing role of information in decision making. We present observations and new results that help rationalize the application of these areas to queueing systems.We prove that the MaxWeight and BackPressure policies are an application of Blackwell's Approachability Theorem. This connects queueing theoretic results with adversarial learning. We then discuss the requirements of statistical learning for service parameter estimation. As an example, we show how queue size regret can be bounded when applying a perceptron algorithm to classify service. Next, we discuss the role of state information in improved decision making. Here we contrast the roles of epistemic information (information on uncertain parameters) and aleatoric information (information on an uncertain state). Finally we review recent advances in the theory of reinforcement learning and queueing, as well as, provide discussion on current research challenges.

show abstract

Augmenting Max-Weight With Explicit Learning for Wireless Scheduling With Switching Costs

Cited by 24 publications

References 28 publications

Learning to Detect an Anomalous Target with Observations from an Exponential Family

Learning to Detect an Anomalous Target with Observations from an Exponential Family

Energy Efficient Priority-Based Task Scheduling for Computation Offloading in Fog Computing

Learning and Information in Stochastic Networks and Queues

Contact Info

Product

Resources

About