RL-Cache: Learning-Based Cache Admission for Content Delivery

Kirilin, Vadim; Sundarrajan, Aditya; Gorinsky, Sergey; Sitaraman, Ramesh K.

doi:10.1109/jsac.2020.3000415

Cited by 57 publications

(32 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The past few years observe the notable progress of deep reinforcement learning(DRL). In the area of network and communication, DRL has been used as a tool to address various problems (e.g., [5,6]). By defining a reasonable reward as the goal, many model-free DRL algorithms can be applied to solve cache problems.…”

Section: Deep Reinforcement Learning Cachementioning

confidence: 99%

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

Yan

Chen

Wang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

The infrastructure for multimedia content delivery has been using more and more edge infrastructure (e.g., base stations, smart routers, etc.), which not only alleviates the centralized servers but also improves the quality of service by letting users access content nearby. Algorithms based on deep reinforcement learning (DRL) have been widely adopted by such edge cache replacement strategies due to their capability to adapt to changing request patterns. However, a DRL cache replacement agent learns extremely slow at an edge cache because of the sparse requests. In this paper, we propose a popularity distillation framework that allows edge caches to refer to content replication strategies of other edge caches. First, we design a collaborative edge cache framework that lets edge caches learn their strategies by handling the local requests using deep reinforcement learning and learn from others by exchanging the "soft" popularity distributions experienced by different edge caches. Second, we design a neighbor maintenance mechanism in which an agent iteratively selects only a small number of neighboring edge caches to perform the collaboration. Experiments driven by a real-world mobile video dataset show that our design can improve the cache hit rate by 3.0% compared with a non-popularity distillation baseline with only a small overhead of transmission data during distillation.

show abstract

Section: Deep Reinforcement Learning Cachementioning

confidence: 99%

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

Yan

Chen

Wang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…One of them performs the investigation in the context of the mobile edge and is introduced above [10]. The second paper focuses on content delivery networks and is titled "Learning-Based Cache Admission for Content Delivery" [12]. In this contribution, Kirilin et al combine Monte Carlo sampling and direct policy search to design RL-Cache, a cacheadmission algorithm, as a simple front end for a CDN server.…”

Section: The Selected Papersmentioning

confidence: 99%

“…Regarding specific machine-learning methods that are introduced to address networking problems, many papers propose reinforcement learning techniques [2], [3], [7], [10]- [13], all of which, except [11], use model-free reinforcement learning. More specifically, [10], [12], and [13] apply a value-based approach, often referred to as Q-learning, while [2] and [3] follow a policy-based Approach; [7] applies a multiarmed bandit model as the basis for reinforcement learning, while all other above-referenced papers rely on Markov Decision Processes (MDPs).…”

Section: The Selected Papersmentioning

confidence: 99%

Guest Editorial Special Issue on Advances in Artificial Intelligence and Machine Learning for Networking

Chemouil

Hui

Kellerer

et al. 2020

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

I. INTRODUCTION A RTIFICIAL Intelligence (AI) and Machine Learning (ML) approaches have emerged in the networking domain with great expectation. They can be broadly divided into AI/ML techniques for network engineering and management, network designs for AI/ML applications, and system concepts. AI/ML techniques for networking and management improve the way we address networking. They support efficient, rapid, and trustworthy engineering, operations, and management. As such, they meet the current interest in softwarization and network programmability that fuels the need for improved network automation in agile infrastructures, including edge and fog environments. Network design and optimization for AI/ML applications addresses the complementary topic of supporting AI/ML-based systems through novel networking techniques, including new architectures and algorithms. The third topic area is system implementation and open-source software development. This evolution draws particular attention to interdisciplinary approaches. Researchers in communication networks apply ML and AI concepts to optimize and automate network architecture, control, and management. Similarly, AI experts collaborate with networking researchers to optimize network support for architecture and design of data communication and processing for AI purposes. This special issue is a follow-up to the JSAC's Special Issue on Artificial Intelligence and Machine Learning for Networking and Communications published in June 2019 [1]. It has been organized by the same core team of researchers.

show abstract

“…A recent trend in networking research is employing machine learning (ML) and, more specifically, deep learning (DL), to inform decision making. Example application domains include video streaming [2,27,49], traffic management [12,57], resource scheduling [28], packet classification [26], caching [22], and congestion control [20,32,62]. In all these contexts, the ability of deep neural networks (DNNs) to pick up on complex patterns in data has proved instrumental.…”

Section: Introductionmentioning

confidence: 99%

Online Safety Assurance for Learning-Augmented Systems

Rotman

Schapira

Tamar

2020

Proceedings of the 19th ACM Workshop on Hot Topics in Networks

View full text Add to dashboard Cite

Recently, deep learning has been successfully applied to a variety of networking problems. A fundamental challenge is that when the operational environment for a learning-augmented system differs from its training environment, such systems often make badly informed decisions, leading to bad performance. We argue that safely deploying learning-driven systems requires being able to determine, in real-time, whether system behavior is coherent, for the purpose of defaulting to a reasonable heuristic when this is not so. We term this the online safety assurance problem (OSAP). We present three approaches to quantifying decision uncertainty that differ in terms of the signal used to infer uncertainty. We illustrate the usefulness of online safety assurance in the context of the proposed deep reinforcement learning (RL) approach to video streaming. While deep RL for video streaming bests other approaches when the operational and training environments match, it is dominated by simple heuristics when the two differ. Our preliminary findings suggest that transitioning to a default policy when decision uncertainty is detected is key to enjoying the performance benefits afforded by leveraging ML without compromising on safety.

show abstract

RL-Cache: Learning-Based Cache Admission for Content Delivery

Cited by 57 publications

References 20 publications

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

DRL-Based Collaborative Edge Content Replication with Popularity Distillation

Guest Editorial Special Issue on Advances in Artificial Intelligence and Machine Learning for Networking

Online Safety Assurance for Learning-Augmented Systems

Contact Info

Product

Resources

About