Semi-Online Computational Offloading by Dueling Deep-Q Network for User Behavior Prediction

Song, Shixin; Fang, Zhiyi; Zhang, Zhanyang; Chen, Chin-Ling; Sun, Hongyu

doi:10.1109/access.2020.3004861

Cited by 18 publications

(13 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [33] considered the communication of terminal devices in an ultradense LAN, where users can select multiple base stations for task offloading to maximize utility performance, reducing the energy consumption of tasks in the computational queue and the channel queue between terminals and base stations, breaking the bottleneck in the high-dimensional space, and achieving the optimal policy based on the DQN algorithm in the case of unknown network dynamic environment. Similarly, the design goal of Song et al [34] is to optimize resource utilization, energy consumption, and network latency by predicting user behavior using a variant of the DQN algorithm to solve the problem. Zou et al [35][36][37] both used the improved Actor-Critic (AC) algorithm in the Deep Deterministic Policy Gradient (DDPG) algorithm to solve the offloading decision policy to balance the workload of the edge server and the final task was reduced in terms of energy consumption and computation time.…”

Section: Offloading Methods With Different Problem Solving Strategiesmentioning

confidence: 99%

Task Offloading Based on LSTM Prediction and Deep Reinforcement Learning for Efficient Edge Computing in IoT

Chen

Yan

et al. 2022

Future Internet

View full text Add to dashboard Cite

In IoT (Internet of Things) edge computing, task offloading can lead to additional transmission delays and transmission energy consumption. To reduce the cost of resources required for task offloading and improve the utilization of server resources, in this paper, we model the task offloading problem as a joint decision making problem for cost minimization, which integrates the processing latency, processing energy consumption, and the task throw rate of latency-sensitive tasks. The Online Predictive Offloading (OPO) algorithm based on Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) networks is proposed to solve the above task offloading decision problem. In the training phase of the model, this algorithm predicts the load of the edge server in real-time with the LSTM algorithm, which effectively improves the convergence accuracy and convergence speed of the DRL algorithm in the offloading process. In the testing phase, the LSTM network is used to predict the characteristics of the next task, and then the computational resources are allocated for the task in advance by the DRL decision model, thus further reducing the response delay of the task and enhancing the offloading performance of the system. The experimental evaluation shows that this algorithm can effectively reduce the average latency by 6.25%, the offloading cost by 25.6%, and the task throw rate by 31.7%.

show abstract

Section: Offloading Methods With Different Problem Solving Strategiesmentioning

confidence: 99%

Task Offloading Based on LSTM Prediction and Deep Reinforcement Learning for Efficient Edge Computing in IoT

Chen

Yan

et al. 2022

Future Internet

View full text Add to dashboard Cite

show abstract

“…problem formulated in (18) will be solved by jointly optimize the offloading policy and computational resource allocation. Specifically, we first divide the original problem into two sub-problems.…”

Section: Problem Formulationmentioning

confidence: 99%

“…Similarly, Ref. [18] uses the DDQN algorithm to predict UEs' offloading modes meanwhile achieving a load balance of the MEC server with unknown environment information, which is interpreted as the unknown channel state information. By adopting the DDQN algorithm, Ref.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Jiang

Gao

et al. 2021

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in device-to-device (D2D)-aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called dueling deep Q-network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme-based DDQN algorithm, a pre-processing procedure is adopted. Then, a distributed deep Q-network (DQN) algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

“…Song et al. [ 19 ] proposed a semi-online computational offloading model soCoM based on dueling deep-Q network to explore the user behaviors in sophisticated action space by reinforcement learning for catching unknown environment information. Liu et al.…”

Section: Related Workmentioning

confidence: 99%

Towards Application-Driven Task Offloading in Edge Computing Based on Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

Edge computing is a new paradigm, which provides storage, computing, and network resources between the traditional cloud data center and terminal devices. In this paper, we concentrate on the application-driven task offloading problem in edge computing by considering the strong dependencies of sub-tasks for multiple users. Our objective is to joint optimize the total delay and energy generated by applications, while guaranteeing the quality of services of users. First, we formulate the problem for the application-driven tasks in edge computing by jointly considering the delays and the energy consumption. Based on that, we propose a novel Application-driven Task Offloading Strategy (ATOS) based on deep reinforcement learning by adding a preliminary sorting mechanism to realize the joint optimization. Specifically, we analyze the characteristics of application-driven tasks and propose a heuristic algorithm by introducing a new factor to determine the processing order of parallelism sub-tasks. Finally, extensive experiments validate the effectiveness and reliability of the proposed algorithm. To be specific, compared with the baseline strategies, the total cost reduction by ATOS can be up to 64.5% on average.

show abstract

Semi-Online Computational Offloading by Dueling Deep-Q Network for User Behavior Prediction

Cited by 18 publications

References 29 publications

Task Offloading Based on LSTM Prediction and Deep Reinforcement Learning for Efficient Edge Computing in IoT

Task Offloading Based on LSTM Prediction and Deep Reinforcement Learning for Efficient Edge Computing in IoT

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Towards Application-Driven Task Offloading in Edge Computing Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About