Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abstract

Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloadingrelated issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybridbased algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.INDEX TERMS Fog computing, Internet of Things (IoT), offloading, reinforcement learning, deep reinforcement learning.

Ripple-Induced Whale Optimization Algorithm for Independent Tasks Scheduling on Fog Computing

“…The size of the input file of a task containing the instructions and its associated data is in the range of [20,100] MB as shown in Table 6. Similarly, the size of the output generated after processing a task is in the range of [5,100] MB. The amount of memory required by each task is in the range of [50,200] MB.…”

“…In a three-layer architecture as shown in Figure 1, the requests received from IoT devices are not directly sent to the cloud, rather they are served at the fog layer, reducing network traffic and load on cloud servers. In case the resources at the fog devices are unable to fulfill the requirements of some tasks, they are sent to the cloud server [5]. Thus, fog computing not only reduces delays and cost but also minimizes overall network congestion.…”

Reinforcement Learning-based Task Scheduling for Heterogeneous Computing in End-Edge-Cloud Environment

“…Over the past few years, reinforcement learning (RL), a subset of machine learning, has proven to be effective for enhancing scheduling algorithms [18]. RL enables an agent to interact with its environment, receive rewards for actions, and learn from these interactions.…”

Bandit Learning-Based Distributed Computation in Fog Computing Networks: A Survey

“…In the context of fog computing, bandit learning algorithms can be used to make optimal offloading decisions by considering the available options and their potential outcomes. By iteratively exploring and exploiting the available options, bandit learning algorithms can provide efficient and effective offloading decisions in dynamic and uncertain fog computing environments [56], [57].…”