Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing

Abstract

Mobile edge computing (a.k.a. fog computing) has recently emerged to enable in-situ processing of delay-sensitive applications at the edge of mobile networks. Providing grid power supply in support of mobile edge computing, however, is costly and even infeasible (in certain rugged or under-developed areas), thus mandating on-site renewable energy as a major or even sole power supply in increasingly many scenarios. Nonetheless, the high intermittency and unpredictability of renewable energy make it very challenging to deliver a high quality of service to users in energy harvesting mobile edge computing systems. In this paper, we address the challenge of incorporating renewables into mobile edge computing and propose an efficient reinforcement learning-based resource management algorithm, which learns on-the-fly the optimal policy of dynamic workload offloading (to the centralized cloud) and edge server provisioning to minimize the long-term system cost (including both service delay and operational cost). Our online learning algorithm uses a decomposition of the (offline) value iteration and (online) reinforcement learning, thus achieving a significant improvement of learning rate and runtime performance when compared to standard reinforcement learning algorithms such as Q-learning.We prove the convergence of the proposed algorithm and analytically show that the learned policy has a simple monotone structure amenable to practical implementation. Our simulation results validate the efficacy of our algorithm, which significantly improves the edge computing performance compared to fixed or myopic optimization schemes and conventional reinforcement learning algorithms.

Collaborative Computation Offloading in Wireless Powered Mobile-Edge Computing Systems

“…3 while |a 1 − a 0 | > ε do 4 if S(λm)<S(γm) then 5 a 0 = λm, a 1 = a 1 , λ m+1 = γm; 6 γ m+1 = a 0 + σ · (a 1 − a 0 ); 7 Update S(λ m+1 ) ← S(γm) and calculate S(γ m+1 ). a 0 = a 0 , a 1 = γm, γ m+1 = λm; 10 λ m+1 = a 0 + (1 − σ) · (a 1 − a 0 ); 11 Update S(γ m+1 ) ← S(λm) and calculate S(λ m+1 ). for both devices.…”

Deep Learning for Hybrid 5G Services in Mobile Edge Computing Systems: Learn From a Digital Twin

“…How to apply machine learning algorithms for user association or task offloading in MEC systems was also studied in some recent works [15][16][17][18]. Deep Q-learning was used to minimize the task execution cost by optimizing offloading decision according to channel state information, queue state information, and energy queue state of the energy harvesting system [15].…”

“…A similar method was also applied for energy harvesting of IoT devices in [16]. The authors of [17] proposed an efficient reinforcement learning-based resource management algorithm to incorporate renewable energy into MEC systems. More recently, a deep reinforcement learning framework for task offloading was studied in a single-AP scenario [18].…”

An online reinforcement learning approach to quality-cost-aware task allocation for multi-attribute social sensing

“…Our work is also related to online learning techniques which have been applied in solving decision making problems in social sensing applications [33,34,35,36,37,38]. In particular, online learning learns to make sequential decisions to achieve the desired quality-of-service of an application and dynamically adjust the learning process based on the streaming data re- [37]. To the best of our knowledge, the QCO-TA scheme is one of the first approaches to leverage online learning techniques to address the quality-cost-aware multi-attribute task allocation problem in social sensing.…”