A Deep Reinforcement Learning Approach for Dynamic Contents Caching in HetNets

Ma, Manyou; Wong, Vincent W. S.

doi:10.48550/arxiv.2004.07911

Cited by 2 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [20] considered a single sensor and proposed to minimize the average AoI subject to the average number of updates. Treating the model parameters of neural networks as transient content, the study in [21] proposed to minimize the average AoI plus cost by using deep Q-network (DQN). Both studies evaluated cache update cost by counting the number of content transmissions.…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

et al. 2020

Preprint

View full text Add to dashboard Cite

Edge nodes (ENs) in Internet of Things commonly serve as gateways to cache sensing data while providing accessing services for data consumers. This paper considers multiple ENs that cache sensing data under the coordination of the cloud. Particularly, each EN can fetch content generated by sensors within its coverage, which can be uploaded to the cloud via fronthaul and then be delivered to other ENs beyond the communication range. However, sensing data are usually transient with time whereas frequent cache updates could lead to considerable energy consumption at sensors and fronthaul traffic loads. Therefore, we adopt age of information to evaluate data freshness and investigate intelligent caching policies to preserve data freshness while reducing cache update costs. Specifically, we model the cache update problem as a cooperative multi-agent Markov decision process with the goal of minimizing the long-term average weighted cost. To efficiently handle the exponentially large number of actions,

show abstract

Section: A Related Workmentioning

confidence: 99%

“…This is reasonable in real applications because data consumers usually have diverse preferences towards content items. Let {N t f,b } f ∈F ,b∈B be the number of user requests received by ENs at epoch t. Consequently, the average AoI to satisfy user demands at epoch t can be calculated as follows [21]:…”

Section: A Age Of Informationmentioning

confidence: 99%

Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…However, it did not take into consideration data freshness. The authors in [9], [10] studied caching transient content items by minimizing the average AoI plus cache update cost. The cost of updating content in these studies was considered as the number of transmissions between sensors and edge nodes.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for IoT Networks: Age of Information and Energy Cost Tradeoff

et al. 2020

Preprint

View full text Add to dashboard Cite

In most Internet of Things (IoT) networks, edge nodes are commonly used as to relays to cache sensing data generated by IoT sensors as well as provide communication services for data consumers. However, a critical issue of IoT sensing is that data are usually transient, which necessitates temporal updates of caching content items while frequent cache updates could lead to considerable energy cost and challenge the lifetime of IoT sensors. To address this issue, we adopt the Age of Information (AoI) to quantify data freshness and propose an online cache update scheme to obtain an effective tradeoff between the average AoI and energy cost. Specifically, we first develop a characterization of transmission energy consumption at IoT sensors by incorporating a successful transmission condition. Then, we model cache updating as a Markov decision process to minimize average weighted cost with judicious definitions of state, action, and reward. Since user preference towards content items is usually unknown and often temporally evolving, we therefore develop a deep reinforcement learning (DRL) algorithm to enable intelligent cache updates. Through trial-and-error explorations, an effective caching policy can be learned without requiring exact knowledge of content popularity. Simulation results demonstrate the superiority of the proposed framework.

show abstract

A Deep Reinforcement Learning Approach for Dynamic Contents Caching in HetNets

Cited by 2 publications

References 16 publications

Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

Deep Reinforcement Learning for IoT Networks: Age of Information and Energy Cost Tradeoff

Contact Info

Product

Resources

About