Age of Information Driven Cache Content Update Scheduling for Dynamic Contents in Heterogeneous Networks

Ma, Manyou; Wong, Vincent W. S.

doi:10.1109/twc.2020.3022895

Cited by 44 publications

(21 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [60], [61] focus on cache update scheduling for age of information (AoI) minimization, in a two-tier HetNet where small cells act as content servers for dynamic content delivery, as shown in Fig. 5.…”

Section: ) Age Of Information Reductionmentioning

confidence: 99%

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

et al. 2022

View full text Add to dashboard Cite

Mobile networks experience a tremendous increase in data volume and user density due to the massive number of coexisting users and devices. An efficient technique to alleviate this issue is to bring the data closer to the users by exploiting cache-aided edge nodes, such as fixed and mobile access points, and even user devices. Meanwhile, the fusion of machine learning and wireless networks offers new opportunities for network optimization when traditional optimization approaches fail or incur high complexity. Among the various machine learning categories, reinforcement learning provides autonomous operation without relying on large sets of historical data for training. In this survey, reinforcement learning-aided mobile edge caching solutions are presented and classified, based on the networking architecture and optimization target. As sixth generation (6G) networks will be characterized by high heterogeneity, fixed cellular, fog, cooperative, vehicular, and aerial networks are studied. The discussion of these works reveals that there exist reinforcement learning-aided caching schemes with varying complexity that can surpass the performance of conventional policy-based approaches. Finally, several open issues are presented, stimulating further interest in this important research field.INDEX TERMS 6G, edge caching, heterogeneous networks, machine learning, mobile edge networks, reinforcement learning.

show abstract

Section: ) Age Of Information Reductionmentioning

confidence: 99%

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Content caching involves static content (such as popular videos) and dynamic content (such as environmental parameters of temperature, humidity, and illuminance). The former is typically not changed in a long period for delay-tolerant applications, and in contrast, the latter is updated in a relatively short period for mission-critical or time-sensitive tasks [56]. Upon storing the reusable contents in the cache resources at the edge networks, the same content for multiple mobile users can be quickly delivered.…”

Section: B Network Uncertaintymentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

et al. 2022

Preprint

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep RL (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. Index Terms-Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL). emote Areas irplane taverse vers Diverse Scenarios MEC Deployment Future Applications Smart City Smart Home Hazard/Remote Areas Smart Agriculture Digital Twin Holographic Communication AR/VR/XR Metaverse Metaverse Metavers Autonomous Driving Small Data Center Switch BS Traffic Light Satellite Ship Airplane S ll S it h BS T ffi Li ht S t llit Shi Ai l 5G Networks and Beyond 5G Networks and Beyond Smart Factory performance. Its typical algorithms are K-means clustering, principal component analysis, and independent component analysis, which can be used in small cell clustering, heterogeneous network clustering, smart grid user classification, etc. However, in unreliable radio network environments, the classification accuracy decreases, which readily causes slow and inaccurate actions in MEC systems. • Different from the static solutions provided by supervised learning and unsupervised learning, RL gives a constantly evolving intelligent framework [21]-[27]. In RL [29], an agent is enabled to make proper decisions based on frequent interactions with the stochastic and dynamic environment. Based on the Markov models, the RL operates in a feedback mechanism (a closed loop) without the knowledge of input data, where based on the previous and current states, the agent can execute its actions to maximize the reward function. Additionally, the above behaviors, including the states, actions, and rewards, accumulate to generate experiences. The classic RL algorithm is Q-learning, which suffers from the curse of dimensionality caused by the large dimensions of the state-action spaces. Accordingly, upon leveraging the low-dimensional representation for the high-dimensional state-ac...

show abstract

“…As terrestrial users in general submit their content requests earlier than the expected time that content files are received, their content requests may be known in advance [25]. For any user i ∈ I, denote by Pr i the probability of content requests of user i received by the content server with…”

Section: Uav Caching Modelmentioning

confidence: 99%

“…• At each time slot t, observe Q i (t), Z i (t), H j (t) for any user i ∈ I, and UAV j ∈ J . • AUxiliary-Tier (AUT) optimization: Choose γ i (t) for each user i to mitigate (25) Minimize…”

Section: B Lyapunov Drift-plus-penaltymentioning

confidence: 99%

Fresh, Fair and Energy-Efficient Content Provision in a Private and Cache-Enabled UAV Network

Yang,

Guo,

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we investigate a private and cacheenabled unmanned aerial vehicle (UAV) network for content provision. Aiming at delivering fresh, fair, and energy-efficient content files to terrestrial users, we formulate a joint UAV caching, UAV trajectory, and UAV transmit power optimization problem. This problem is confirmed to be a sequential decision problem with mixed-integer non-convex constraints, which is intractable directly. To this end, we propose a novel algorithm based on the techniques of subproblem decomposition and convex approximation. Particularly, we first propose to decompose the sequential decision problem into multiple repeated optimization subproblems via a Lyapunov technique. Next, an iterative optimization scheme incorporating a successive convex approximation (SCA) technique is explored to tackle the challenging mixed-integer non-convex subproblems. Besides, we analyze the convergence of the proposed algorithm and derive the theoretical value of the expected peak age of information (PAoI) to estimate the content freshness. Simulation results demonstrate that the proposed algorithm can achieve the expected PAoI close to the theoretical value and is more 22.11% and 70.51% energyefficient and fairer than benchmark algorithms.

show abstract

Age of Information Driven Cache Content Update Scheduling for Dynamic Contents in Heterogeneous Networks

Cited by 44 publications

References 31 publications

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Fresh, Fair and Energy-Efficient Content Provision in a Private and Cache-Enabled UAV Network

Contact Info

Product

Resources

About