Intelligent Video Caching at Network Edge: A Multi-Agent Deep Reinforcement Learning Approach

Wang, Fangxin; Wang, Rui; Liu, Jiangchuan; Shea, Ryan; Sun, Lifeng

doi:10.1109/infocom41043.2020.9155373

Cited by 103 publications

(35 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An additional work, focusing on QoE improvement and the reduction of latency for users requesting video content and backhaul usage is presented in [94]. Here, multi-agent AC DRL-based caching is developed, treating each network edge, as a cooperative learning agent and avoiding the large action spaces of centralized single-agent approaches.…”

Section: ) Qoe Improvementmentioning

confidence: 99%

“…Single-agent DRL-aided caching has been proposed in [113], [114] where a single edge node makes suitable caching decisions. This process requires from every single node to have its own caching policy and a single central agent to make the global decisions resulting in a huge action space [94]. CognitiveCache offers better convergence than [94] while comparisons with DQNCache [114], Prob-Cache [115], LRU and LFU show that it reduces latency by 33%, 47%, 66%, 71% and transmission cost by 23%, 75%, 83% and 87%, respectively.…”

Section: ) Delay Reductionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

et al. 2022

View full text Add to dashboard Cite

Mobile networks experience a tremendous increase in data volume and user density due to the massive number of coexisting users and devices. An efficient technique to alleviate this issue is to bring the data closer to the users by exploiting cache-aided edge nodes, such as fixed and mobile access points, and even user devices. Meanwhile, the fusion of machine learning and wireless networks offers new opportunities for network optimization when traditional optimization approaches fail or incur high complexity. Among the various machine learning categories, reinforcement learning provides autonomous operation without relying on large sets of historical data for training. In this survey, reinforcement learning-aided mobile edge caching solutions are presented and classified, based on the networking architecture and optimization target. As sixth generation (6G) networks will be characterized by high heterogeneity, fixed cellular, fog, cooperative, vehicular, and aerial networks are studied. The discussion of these works reveals that there exist reinforcement learning-aided caching schemes with varying complexity that can surpass the performance of conventional policy-based approaches. Finally, several open issues are presented, stimulating further interest in this important research field.INDEX TERMS 6G, edge caching, heterogeneous networks, machine learning, mobile edge networks, reinforcement learning.

show abstract

Section: ) Qoe Improvementmentioning

confidence: 99%

Section: ) Delay Reductionmentioning

confidence: 99%

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Enabling edge cache collaboration is a critical practice to improve content replication performance. Wang et al [8] propose 'MacoCache', a multi-agent DRL-based algorithm to minimize both content access latency and traffic cost by letting edge learns its policy in conjunction with other edges. In our scenario, we proposed a dual-RL architecture that incorporates knowledge distillation [3] for edge-network cache collaboration, one for cache replacement, and one for collaboration.…”

Section: Collaboration In Edge Cachementioning

confidence: 99%

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

Yan

Chen

Wang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

The infrastructure for multimedia content delivery has been using more and more edge infrastructure (e.g., base stations, smart routers, etc.), which not only alleviates the centralized servers but also improves the quality of service by letting users access content nearby. Algorithms based on deep reinforcement learning (DRL) have been widely adopted by such edge cache replacement strategies due to their capability to adapt to changing request patterns. However, a DRL cache replacement agent learns extremely slow at an edge cache because of the sparse requests. In this paper, we propose a popularity distillation framework that allows edge caches to refer to content replication strategies of other edge caches. First, we design a collaborative edge cache framework that lets edge caches learn their strategies by handling the local requests using deep reinforcement learning and learn from others by exchanging the "soft" popularity distributions experienced by different edge caches. Second, we design a neighbor maintenance mechanism in which an agent iteratively selects only a small number of neighboring edge caches to perform the collaboration. Experiments driven by a real-world mobile video dataset show that our design can improve the cache hit rate by 3.0% compared with a non-popularity distillation baseline with only a small overhead of transmission data during distillation.

show abstract

“…Deep reinforcement learning has been adopted in [16], while their model confines service to identically sized content, and only minimizes the traffic cost. Zeng et al [17] utilize primal-dual decomposition to translate the joint problem into bi-problems, and design an approximation algorithm to solve each sub-problem.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Heuristic algorithms without provable performance guarantee are proposed in [13]- [15]. Among those algorithms with theoretical approximation guarantees [16]- [18], the performances are usually unsatisfactory.…”

mentioning

confidence: 99%

Online Service Placement and Request Scheduling in MEC Networks

Su¹,

Wang²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Mobile edge computing (MEC) emerges as a promising solution for servicing delay-sensitive tasks at the edge network. A body of recent literature started to focus on cost-efficient service placement and request scheduling. This work investigates the joint optimization of service placement and request scheduling in a dense MEC network, and develops an efficient online algorithm that achieves closeto-optimal performance. Our online algorithm consists of two basic modules: (1) a regularization with look-ahead approach from competitive online convex optimization, for decomposing the offline relaxed minimization problem into multiple sub-problems, each of which can be efficiently solved in each time slot; (2) a randomized rounding method to transform the fractional solution of offline relaxed problem into integer solution of the original minimization problem, guaranteeing a low competitive ratio. Both theoretical analysis and simulation studies corroborate the efficacy of our proposed online MEC optimization algorithm.

show abstract

Intelligent Video Caching at Network Edge: A Multi-Agent Deep Reinforcement Learning Approach

Cited by 103 publications

References 27 publications

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

Online Service Placement and Request Scheduling in MEC Networks

Contact Info

Product

Resources

About