Learning-Based Decentralized Offloading Decision Making in an Adversarial Environment

Abstract

Vehicular fog computing (VFC) pushes the cloud computing capability to the distributed fog nodes at the edge of the Internet, enabling compute-intensive and latency-sensitive computing services for vehicles through task offloading. However, a heterogeneous mobility environment introduces uncertainties in terms of resource supply and demand, which are inevitable bottlenecks for the optimal offloading decision. Also, these uncertainties bring extra challenges to task offloading under the oblivious adversary attack and data privacy risks. In this article, we develop a new adversarial online learning algorithm with bandit feedback based on the adversarial multi-armed bandit theory, to enable scalable and low-complexity offloading decision making. Specifically, we focus on optimizing fog node selection with the aim of minimizing the offloading service costs in terms of delay and energy. The key is to implicitly tune the exploration bonus in the selection process and the assessment rules of the designed algorithm, taking into account volatile resource supply and demand. We theoretically prove that the input-size dependent selection rule allows to choose a suitable fog node without exploring the sub-optimal actions, and also an appropriate score patching rule allows to quickly adapt to evolving circumstances, which reduce variance and bias simultaneously, thereby achieving a better exploitation-exploration balance. Simulation results verify the effectiveness and robustness of the proposed algorithm.Index Terms-Vehicular fog computing, task offloading, online learning, adversarial multi-armed bandit.Byungjin Cho received the doctoral degree in communications engineering from Aalto University, in 2016. He is currently a postdoctoral researcher with the Department of Communications and Networking, Aalto University. His research interests include resource managements in networked systems using algorithmic decision theory.Yu Xiao Yu Xiao received the doctoral degree in computer science from Aalto University, in 2012. She is currently an assistant professor with the Department of Communications and Networking, Aalto University. Her current research interests include edge computing, wearable sensing and extended reality. She is a member of the IEEE.

Quantum bandit with amplitude amplification exploration in an adversarial environment

“…A service client (SC) generates tasks, while a set of service providers (SPs) k ∈ K = {1, ..., K} execute the requested tasks with their own available resources. An SC can send a task, e.g., offloading a computational task [3], t to any SP k within its accessible communication range. Here k ∈ K t ⊆ K where K t is a candidate SP set.…”

“…The cost for offloading a task includes the cost for uploading the input to an SP k, and the execution cost at the SP, downloading the result to the SC, expressed as D t k . Assuming that the feedback size is small enough that the down-link cost can be ignored [3].…”

“…, but it could cause large fluctuation in the cost due to inverse-proportion to p t k . Instead, this work considers Exp3 algorithm endowed with implicit exploration (IX)-style cost estimates [3], which controls the variance at the price of extra bias. After each action, the cost estimate is calculated as…”

“…where U (φ1,m) is an operation based on an oracle query, shifting the phase of the target action 3 2 Amplitudes are quantum probabilities representing the chance that a quantum state will be observed when being collapsed. 3 Classically, m = arg max k p k , while non-classically done by [15].…”

“…Enabling such emerging applications to work in a pervasive uncertain environment mandates the need for intelligent decisionmaking (DM) to choose an ideally suited computing server guaranteeing the quality of service, e.g., offloaded to nodes geared with sufficiently powerful computing capability [2]. To solve the suitable provider identification problem, sequential DM has been leveraged for its ability to learn in a trial/error fashion without explicit knowledge of the environment, while facing the exploration/exploitation (ExR/ExT) dilemma [3]. The exploration strategy is known as a crucial ingredient for learning-based DM: under-ExR makes the decision stick at a sub-optimal strategy, while over-ExR may incur an ExR cost.…”