A Smart Cache Content Update Policy Based on Deep Reinforcement Learning

Abstract

This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%–64% improved in terms of the cache hit ratio and 56%–59% decreased in terms of the average latency.

Cluster-Based Multi-User Multi-Server Caching Mechanism in Beyond 5G/6G MEC

“…Ref. [ 38 ] studied applying the Deep Q-learning Network (DQN) algorithm to increase the hit ratio and minimize time delay. However, it is assumed that all contents have an equal size (2000 bits) and servers’ caching capacity is sufficient to carry all of the favored contents.…”

Analysis and Application of Practical Ability of Multimedia Technology in Art Education

“…According to the system model, the location distribution of DUEs obeys the two-dimensional HPPP with parameter  , then for a single DUE, the number of other DUEs within its D2D effective transmission range satisfies the Poisson distribution with parameter  . A collaborative hit occurs when a DUE makes a request for file c f and there is at least one user within its D2D effective transmission range who has cached file c f [28]. Considering the chi-square nature of the DUE location distribution, the distribution of DUEs that have cached file…”

Edge Caching and Computing for Wireless Networks

“…In addition to the above works, there are some rest papers in this special issue on the application of artificial intelligence on the wireless caching and computing networks, as shown in Refs. [4][5][6]. In particular, deep reinforcement learning was proposed in these works, in order to provide an intelligent solution to the system resource allocation, such as caching allocation and offloading allocation, bandwidth allocation, and power allocation.…”