Deep reinforcement learning‐based joint optimization of computation offloading and resource allocation in F‐RAN

Abstract

The fog radio access network (F-RAN) has been regarded as a promising wireless access network architecture in the fifth generation (5G) and beyond systems to satisfy the increasing requirements for low-latency and high-throughput services by providing fog computing. However, because the cloud computing centre and fog computing-enabled access points (F-APs) in the F-RAN have different computation and communication capabilities, it is crucial to make an efficient computation offloading and resource allocation strategy that can fully exploit the potential of the F-RAN system. In this paper, the authors investigate a decentralized low-complexity deep reinforcement learning (DRL)based framework for joint computation task offloading and resource allocation in the F-RAN, which supports assistive computing-enabled tasks offloading between F-APs. Considering the constraints of task latency, wireless transmission rate, transmission power, and computational resource capacity, the authors formulate the system processing efficiency maximization problem by jointly optimizing offloading mode selection, channel allocation, power control, and computation resource allocation in the F-RAN. To solve this non-linear and non-convex problem, the authors propose a federated DRL-based computation offloading and resource allocation algorithm to improve the task processing efficiency and ensure privacy in the system, which can significantly reduce the computing complexity and signalling overhead of the training process compared with the centralized learning-based method. Specifically, each local F-AP agent consists of dueling deep Q-network (DDQN) and deep deterministic policy gradient (DDPG) networks to appropriately deal with discrete and continuous valuable action spaces, respectively. Finally, the simulation results show that the proposed federated DRL algorithm can achieve significant performance improvements in terms of system processing efficiency and task latency compared with other benchmarks.

Machine-learning-based wireless network resource allocation strategy

“…Techniques like Target Networks and gradient clipping enhance stability and speed. After 2000 episodes, this approach outperforms traditional Q-learning by over 15%, demonstrating significant improvements in resource allocation efficiency [7].As shown in Figure 2.…”

Motor Interaction Control Based on Muscle Force Model and Depth Reinforcement Strategy

“…In order to optimize the energy utilization efficiency of electric buses and extend the power system life, Huang et al proposed to construct an energy management model, validated the effectiveness of the model, and found that the model effectively extended the life of the battery and improved the efficiency of energy utilization, which, in turn, reduced the total operating cost, and the model made a contribution and had practical application value [ 11 , 12 , 13 ]. To optimize the computational performance and resource allocation capacity of a fog computing wireless access network, Jo et al proposed to construct a computational task offloading and resource allocation strategy, validated the effectiveness of the proposed optimization strategy, and found that this strategy significantly improved the system processing efficiency and resource allocation compared with the traditional processing methods [ 14 ]. In the transportation system, the logistics path-planning performance is insufficient and there is the problem of long computation times, so Yu et al proposed the deep reinforcement learning mechanism to optimize the logistics path-planning model to verify the model’s effectiveness, and found that the computation time of the model compared with the traditional model was significantly shortened, and the phase of the computation time in the path planning was better [ 15 ].…”

DRL-Based Computation Offloading and Resource Allocation in Green MEC-Enabled Maritime-IoT Networks

“…Herein, because discrete and continuous variables are intermingled and continuous variables have infinite values, the DQN scheme cannot be utilized directly. The size of the action space increases exponentially as the discrete level expands, leading to a reduction in performance since discretizing a continuous variable into a finite level will result in quantization error [48]. To address the problem of continuous variables, we developed the DDPG-based algorithm.…”