Stochastic Game Based Cooperative Alternating Q-Learning Caching in Dynamic D2D Networks

Zhang, Tiankui; Fang, Xinyuan; Wang, Ziduan; Liu, Yuanwei; Nallanathan, Arumugam

doi:10.1109/tvt.2021.3120292

Cited by 14 publications

(7 citation statements)

References 44 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Literature [20] adopts the Stackelberg game to optimize UA, power allocation of non-orthogonal multiple access (NOMA), unmanned aerial vehicle (UAV) deployment and caching placement to minimize the content delivery delay. In [21], the authors improve the content caching and sharing of D2D networks by a CAQL-based caching placement algorithm. Taking into account Coordinated MultiPoint (CoMP) joint transmission technique, a reinforcement learning (RL)based algorithm is presented in [22] to maximize the delay reduction.…”

Section: A Related Workmentioning

confidence: 99%

Deep Q-Learning Aided Energy-Efficient Caching and Transmission for Adaptive Bitrate Video Streaming Over Dynamic Cellular Networks

Xie

2024

IEEE Access

View full text Add to dashboard Cite

Adaptive bitrate video streaming (ABRVS) and edge caching are two techniques that hold the potential to improve user-perceived video viewing experience. In this paper, we investigate the content caching, transcoding and transmission for ABRVS in cache-enabled cellular networks. Considering the dynamic characteristics of video popularity distribution and wireless network environment, to improve energy efficiency and minimize system energy consumption, we begin by formulating a long-term optimization problem that focuses on both video caching and user association (UA). The problem is then transformed into a Markov decision process (MDP), which is solved by designing a deep Q-learning network (DQN)based algorithm. Using this algorithm, we can obtain the optimal video caching and UA solutions. Since the action space of the MDP is huge, to cope with the "curse of dimensionality", linear approximation is integrated into the designed algorithm. Finally, the proposed algorithm's convergence and effectiveness in reducing long-term system energy consumption are demonstrated through extensive simulations.INDEX TERMS Adaptive bitrate video streaming; edge caching; energy efficiency; deep Q-learning.

show abstract

Section: A Related Workmentioning

confidence: 99%

Deep Q-Learning Aided Energy-Efficient Caching and Transmission for Adaptive Bitrate Video Streaming Over Dynamic Cellular Networks

Xie

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Nadia Abdolkhani et al 25 propose a close to optimal low complexity heuristic cache placement policy To solve the users' equipment (UE) cache memory sizes inconsistency problem. Zhang et al 26 model the dynamic network in a D2D caching cellular network with content popularity distribution and user terminal location time‐varying characteristics as a stochastic game to design a cooperative cache placement strategy. To solve the problem of randomness of benefits and ensure that benefits are equal for each user terminal (UT), Zhang et al 27 propose a multiwinner once auction‐based caching (MOAC) placement algorithm to maximize the content sharing revenue of all the UTs.…”

Section: Related Workmentioning

confidence: 99%

Content sharing strategy for blind popularity distribution in D2D communications

Zhuang

Song²,

Chen

et al. 2023

Trans Emerging Tel Tech

View full text Add to dashboard Cite

The development of multimedia content continuously encourages the appearances of new multimedia applications. Meanwhile, Device to Device (D2D) is regarded as an important 5G technology that creates a direct connection between two mobile devices. We combine the content sharing traits in D2D network situations in response to the problem of massive amounts of multimedia content being distributed. As a result, in this paper, we provide a popularity‐based information mining and content placement strategy for blind popularity distribution in D2D scenarios. To analyze the cache hit performance in D2D networks under the presumption of deterministic content popularity, we first construct a D2D content caching framework. Then, we design a multiarm bandits model and suggest a single and multicache placement policy based on online learning for blind popularity in D2D networks. Finally, the experimental results demonstrate that the proposed method achieves better convergence and a better cache hit ratio than the other strategies.

show abstract

“…FDC agents are used in [42], assuming an offline training phase, shared state, and common reward. Edge caching in [43,44] is also treated with an FDC algorithm, sharing a global state between agents, with the difference being that it is compressed through learning in order to minimize communication costs.…”

Section: Edge Cachingmentioning

confidence: 99%

Distributed Machine Learning and Native AI Enablers for End-to-End Resources Management in 6G

Karachalios,

Zafeiropoulos,

Kontovasilis

et al. 2023

Electronics

View full text Add to dashboard Cite

6G targets a broad and ambitious range of networking scenarios with stringent and diverse requirements. Such challenging demands require a multitude of computational and communication resources and means for their efficient and coordinated management in an end-to-end fashion across various domains. Conventional approaches cannot handle the complexity, dynamicity, and end-to-end scope of the problem, and solutions based on artificial intelligence (AI) become necessary. However, current applications of AI to resource management (RM) tasks provide partial ad hoc solutions that largely lack compatibility with notions of native AI enablers, as foreseen in 6G, and either have a narrow focus, without regard for an end-to-end scope, or employ non-scalable representations/learning. This survey article contributes a systematic demonstration that the 6G vision promotes the employment of appropriate distributed machine learning (ML) frameworks that interact through native AI enablers in a composable fashion towards a versatile and effective end-to-end RM framework. We start with an account of 6G challenges that yields three criteria for benchmarking the suitability of candidate ML-powered RM methodologies for 6G, also in connection with an end-to-end scope. We then proceed with a focused survey of appropriate methodologies in light of these criteria. All considered methodologies are classified in accordance with six distinct methodological frameworks, and this approach invites broader insight into the potential and limitations of the more general frameworks, beyond individual methodologies. The landscape is complemented by considering important AI enablers, discussing their functionality and interplay, and exploring their potential for supporting each of the six methodological frameworks. The article culminates with lessons learned, open issues, and directions for future research.

show abstract

Stochastic Game Based Cooperative Alternating Q-Learning Caching in Dynamic D2D Networks

Cited by 14 publications

References 44 publications

Deep Q-Learning Aided Energy-Efficient Caching and Transmission for Adaptive Bitrate Video Streaming Over Dynamic Cellular Networks

Deep Q-Learning Aided Energy-Efficient Caching and Transmission for Adaptive Bitrate Video Streaming Over Dynamic Cellular Networks

Content sharing strategy for blind popularity distribution in D2D communications

Distributed Machine Learning and Native AI Enablers for End-to-End Resources Management in 6G

Contact Info

Product

Resources

About