Computation offloading time optimisation via Q‐learning in opportunistic edge computing

Yang, Guisong; Hou, Ling; Cheng, Hao; He, Xingyu; He, Daojing; Chan, Sammy

doi:10.1049/iet-com.2020.0765

Cited by 8 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To reduce the overload probability and processing time, the proposed methods enable fog nodes to choose an available neighboring fog node based on their resource capabilities and to offload the maximum number of incoming tasks. Similarly, in [83], a Q-learning-based technique for selecting ideal offloading nodes in opportunistic edge computing was proposed.…”

Section: A Value-based Algorithmsmentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloadingrelated issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybridbased algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.INDEX TERMS Fog computing, Internet of Things (IoT), offloading, reinforcement learning, deep reinforcement learning.

show abstract

Section: A Value-based Algorithmsmentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In this field, only a limited number of studies have tackled the challenges of service placement or user traffic routing, specifically specially focusing on certain aspects of the heterogeneity of fog nodes [32][33][34][35][36][37][38]. However, none of the existing research has addressed the problem of user traffic routing and service placement in the fog computing layer, aiming to minimize the service provisioning cost, particularly in the presence of ad hoc and fixed (dedicated) fog computing nodes.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Service Provisioning in Heterogenous Fog Computing Architecture Using Deep Reinforcement Learning

Govarchinghaleh,

Sabaei

2024

Preprint

View full text Add to dashboard Cite

The exponential growth of IoT devices and the surge in the data volume, coupled with the rise of latency-intensive applications, has led to a heightened interest in Fog computing to meet user demands. In this context, the service provisioning problem consists of dynamically selecting desirable fog computing nodes and routing user traffic to these nodes. Given that the fog computing layer is composed of heterogeneous nodes, which vary in resource capacity, availability, and power sources, the service provisioning problem becomes challenging. Existing solutions, often using classical optimization approaches or heuristics algorithms due to the NP-hardness of the problem, have struggled to address the issue effectively, particularly in accounting for the heterogeneity of fog nodes and uncertainty of the ad hoc fog nodes. These techniques show exponential computation times and deal only with small network scales. To overcome these issues, we are motivated to replace these approaches with Deep Reinforcement Learning (DRL) techniques, specifically employing the Proximal Policy Optimization (PPO) algorithm to understand the dynamic behavior of the environment. The main objective of the proposed DRL-based dynamic service provisioning (DDSP) algorithm is minimizing service provisioning costs while considering service delay constraints, the uncertainty of ad hoc fog nodes, and the heterogeneity of both ad hoc and dedicated fog nodes. Extensive simulations demonstrate that our approach provides a near-optimal solution with high efficiency. Notably, our proposed algorithm selects more stable fog nodes for service provisioning and successfully minimizes cost even with uncertainty regarding ad hoc fog nodes, compared to heuristic algorithms.

show abstract

Machine learning-based computation offloading in edge and fog: a systematic review

Taheri-abed,

Eftekhari Moghadam,

Rezvani

2023

Cluster Comput

View full text Add to dashboard Cite

Computation offloading time optimisation via Q‐learning in opportunistic edge computing

Cited by 8 publications

References 32 publications

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Dynamic Service Provisioning in Heterogenous Fog Computing Architecture Using Deep Reinforcement Learning

Machine learning-based computation offloading in edge and fog: a systematic review

Contact Info

Product

Resources

About