Efficient Content Replacement in Wireless Content Delivery Network with Cooperative Caching

Sung, Jihoon; Kim, Kyounghye; Kim, Junhyuk; Rhee, June-Koo Kevin

doi:10.1109/icmla.2016.0096

Cited by 15 publications

(8 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, a deep actor-critic framework for content caching at the base station is proposed in [10]. And to seek for the optimal cooperative caching policy in a decentralized caching network, the authors in [11] presented a multi-armed bandit based solution, while the authors in [12] adopted a multi-agent Q-learning scheme.…”

Section: Introductionmentioning

confidence: 99%

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

Zhong

Gursoy

Velipasalar

2019

ICC 2019 - 2019 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

The growing demand on high-quality and lowlatency multimedia services has led to much interest in edge caching techniques. Motivated by this, we in this paper consider edge caching at the base stations with unknown content popularity distributions. To solve the dynamic control problem of making caching decisions, we propose a deep actor-critic reinforcement learning based multi-agent framework with the aim to minimize the overall average transmission delay. To evaluate the proposed framework, we compare the learning-based performance with three other caching policies, namely least recently used (LRU), least frequently used (LFU), and first-in-first-out (FIFO) policies. Through simulation results, performance improvements of the proposed framework over these three caching algorithms have been identified and its superior ability to adapt to varying environments is demonstrated.Index Terms-Deep reinforcement learning, multi-agent learning, edge caching.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

Zhong

Gursoy

Velipasalar

2019

ICC 2019 - 2019 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

show abstract

“…Wireless content delivery and replacement are the focus of [126]. Multi-agent Q-learning is used, modeling the problem as an MDP, where each small BS is considered as an agent.…”

Section: ) Caching Efficiencymentioning

confidence: 99%

“…This study is highly related to [128] which also models content replacement as an MDP. However, the two works aim at different objectives, since in [126] the goal is to maximize the cache hit rate while the objective of [128] is to minimize the system transmission cost. Moreover, the work in [126] adopts more realistic simulation parameters and network procedures, as multiple contents can be simultaneously replaced.…”

Section: ) Caching Efficiencymentioning

confidence: 99%

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

et al. 2022

View full text Add to dashboard Cite

Mobile networks experience a tremendous increase in data volume and user density due to the massive number of coexisting users and devices. An efficient technique to alleviate this issue is to bring the data closer to the users by exploiting cache-aided edge nodes, such as fixed and mobile access points, and even user devices. Meanwhile, the fusion of machine learning and wireless networks offers new opportunities for network optimization when traditional optimization approaches fail or incur high complexity. Among the various machine learning categories, reinforcement learning provides autonomous operation without relying on large sets of historical data for training. In this survey, reinforcement learning-aided mobile edge caching solutions are presented and classified, based on the networking architecture and optimization target. As sixth generation (6G) networks will be characterized by high heterogeneity, fixed cellular, fog, cooperative, vehicular, and aerial networks are studied. The discussion of these works reveals that there exist reinforcement learning-aided caching schemes with varying complexity that can surpass the performance of conventional policy-based approaches. Finally, several open issues are presented, stimulating further interest in this important research field.INDEX TERMS 6G, edge caching, heterogeneous networks, machine learning, mobile edge networks, reinforcement learning.

show abstract

“…Lu et al [24] proposed a session-based cloud video delivery network framework for the dynamic characteristics of mobile users in cloud video delivery networks which effectively reduced the cost and improved the performance of the algorithm. Documents [25] and [26] propose efficient cache placement strategy in wireless content delivery networks. Zheng and Zheng [27] propose an improved heuristic genetic algorithm for static content delivery in cloud storage and obtain optimal content delivery program.…”

Section: Related Workmentioning

confidence: 99%

Q-Learning Based Content Placement Method for Dynamic Cloud Content Delivery Networks

Liu¹,

Lu²,

Zhang³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

How to reduce the content placement cost of cloud content delivery networks (CCDNs) is a hot topic in recent years. Traditional content placement methods mainly reduce the cost of content placement by constructing delivery trees, but they cannot adapt to the dynamic deployment of cloud proxy servers in the CCDNs. In addition, the traditional content placement method only provides delivery paths according to local decision-making without considering global dynamics of the congestion in the CCDNs, which is also one of the main factors causing high cost of content placement. To solve these problems, we propose a content placement model based on Q-learning for the dynamic CCDNs, called Q-content placement model (Q-CPM). This Q-learning approach can lead to better routing decisions due to up-to-date and more reliable congestion values. Then, based on the Q-CPM model, an algorithm is proposed to construct the Q-adaptive delivery tree (Q-ADT). In this algorithm, local and nonlocal congestion information is propagated over network learning packets. Through this algorithm, the paths with low congestion cost will be selected and can adapt to the dynamic cloud delivery environment. The experimental results show that the method can adapt to the dynamic changes of the CCDNs flexibly and reduce the overall congestion cost of content placement effectively.

show abstract

Efficient Content Replacement in Wireless Content Delivery Network with Cooperative Caching

Cited by 15 publications

References 6 publications

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

Q-Learning Based Content Placement Method for Dynamic Cloud Content Delivery Networks

Contact Info

Product

Resources

About