Deep Reinforcement Learning-Based Content Placement and Trajectory Design in Urban Cache-Enabled UAV Networks

Abstract

Cache-enabled unmanned aerial vehicles (UAVs) have been envisioned as a promising technology for many applications in future urban wireless communication. However, to utilize UAVs properly is challenging due to limited endurance and storage capacity as well as the continuous roam of the mobile users. To meet the diversity of urban communication services, it is essential to exploit UAVs’ potential of mobility and storage resource. Toward this end, we consider an urban cache-enabled communication network where the UAVs serve mobile users with energy and cache capacity constraints. We formulate an optimization problem to maximize the sum achievable throughput in this system. To solve this problem, we propose a deep reinforcement learning-based joint content placement and trajectory design algorithm (DRL-JCT), whose progress can be divided into two stages: offline content placement stage and online user tracking stage. First, we present a link-based scheme to maximize the cache hit rate of all users’ file requirements under cache capacity constraint. The NP-hard problem is solved by approximation and convex optimization. Then, we leverage the Double Deep Q-Network (DDQN) to track mobile users online with their instantaneous two-dimensional coordinate under energy constraint. Numerical results show that our algorithm converges well after a small number of iterations. Compared with several benchmark schemes, our algorithm adapts to the dynamic conditions and provides significant performance in terms of sum achievable throughput.

A bi-objective optimization model for the UAV caching at amusement parks in fog computing

“…., f L }, L denotes the library size. In MPC, each UAV dependently caches N most popular contents [33]. In MPC, if the content cached in the UAV is popular, q = 1, otherwise q = 0.…”

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

“…Another RL-based solution for improved energy efficiency in cache-enabled UAV-aided networks is presented in [172]. Focusing on an urban scenario with mobile users, the storage and energy capabilities of the UAV are considered, towards maximizing the sum achievable throughput.…”

“…through VFA [165]. Other works have developed joint content placement and trajectory design solutions [171], [172] through DRL with promising performance. However, currently, hybrid learningbased solutions, combining offline training using historical data and online RL-aided operation for faster convergence and UAV deployment are missing.…”

UAV Path Planning Based on Multicritic-Delayed Deep Deterministic Policy Gradient

“…In recent years, unmanned aerial vehicles (UAVs) have been widely applied, and their high maneuverability and rapidly deployable UAVs have been applied to search and rescue [1], multi-UAV cooperation [2], formation flight [3], remote surveillance [4], and other fields [5][6][7]. UAV faces a variety of complex challenges and complicated tasks.…”