Caching Placement and Resource Allocation for Cache-Enabling UAV NOMA Networks

Abstract

This article investigates the cache-enabling unmanned aerial vehicle (UAV) cellular networks with massive access capability supported by non-orthogonal multiple access (NOMA). The delivery of a large volume of multimedia contents for ground users is assisted by a mobile UAV base station, which caches some popular contents for wireless backhaul link traffic offloading. In cache-enabling UAV NOMA networks, the caching placement of content caching phase and radio resource allocation of content delivery phase are crucial for network performance. To cope with the dynamic UAV locations and content requests in practical scenarios, we formulate the longterm caching placement and resource allocation optimization problem for content delivery delay minimization as a Markov decision process (MDP). The UAV acts as an agent to take actions for caching placement and resource allocation, which includes the user scheduling of content requests and the power allocation of NOMA users. In order to tackle the MDP, we propose a Q-learning based caching placement and resource allocation algorithm, where the UAV learns and selects action with soft εgreedy strategy to search for the optimal match between actions and states. Since the action-state table size of Q-learning grows with the number of states in the dynamic networks, we propose a function approximation based algorithm with combination of stochastic gradient descent and deep neural networks, which is suitable for large-scale networks. Finally, the numerical results show that the proposed algorithms provide considerable performance compared to benchmark algorithms, and obtain a tradeoff between network performance and calculation complexity.Index termsdynamic resource allocation, non-orthogonal multiple access, reinforcement learning, unmanned aerial vehicle I. INTRODUCTIONWith the explosion of massive multimedia applications and the continuous growth of mobile data traffic, wireless communication faces the problem of limited resources. In order to effectively meet the increasing user demand for high data rate and low access delay, many works [1-5] have paid attention to wireless connectivity from the sky with unmanned aerial vehicles (UAVs). UAVs, also known as remotely piloted aircraft systems (RPAS) or drones, are small pilotless aircrafts

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

“…In cache-enabled UAV-aided NOMA networks, the authors in [165] turn to RL for leveraging the dynamic UAV mobility and content request variations. Initially, long-term optimization of cache placement, user scheduling and NOMA power allocation is formulated, minimizing the sum delay of ground users.…”

“…As a remedy, RL methods provide an alternative approach to achieve near-optimal operation, as long as the issue of large action and state space is addressed, e.g. through VFA [165]. Other works have developed joint content placement and trajectory design solutions [171], [172] through DRL with promising performance.…”

Trajectory Optimization for UAV Emergency Communication With Limited User Equipment Energy: A Safe-DQN Approach

“…Due to the high flexible mobility, UAV has attracted significant research interest in the field of wireless communication [7]. There are many researches that combine UAV with different communication technologies, such as non-orthogonal multiple access [8][9][10], massive MIMO [11], millimeter wave communication [12] and reconfigurable intelligent surfaces [13]. Meanwhile, caching-enabled UAV cellular networks has attracted increasing attention to effectively alleviate the traffic load of wireless backhaul links [14,15].…”

Multi Agent Reinforcement Learning Trajectory Design and Two-Stage Resource Management in CoMP UAV VLC Networks

“…Every UAV possesses an independent deep Q-network (DQN) for its own action strategy, while the rest of the UAVs are considered parts of the environment. Specifically, the authors of [23] develop an architecture for delivering content to ground users in a hotspot area using UAV NOMA cellular networks enabling caching. By optimizing the caching placement of a UAV, the scheduling of content requests by users, and the power allocation of NOMA users, they formulate an optimization problem to minimize content delivery delay as a Markov decision process (MDP).…”