Cooperative Caching and Fetching in D2D Communications - A Fully Decentralized Multi-Agent Reinforcement Learning Approach

Yan, Yan; Zhang, Baoxian; Li, Cheng; Su, Changqing

doi:10.1109/tvt.2020.3042089

Cited by 28 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Computer Networks [13,14] Electronics [15] IEEE Access [16,17] IEEE Communications Letters [18,19,20,21] IEEE Internet of Things Journal [22] IEEE Journal on Selected Areas in Communications [23,24,25,26] IEEE Systems Journal [27] IEEE Transactions on Industrial Informatics [28] IEEE Transactions on Information Forensics and Security [29] IEEE Transactions on Mobile Computing [30,31] IEEE Transactions on Network Science and Engineering [32] IEEE Transactions on Network and Service Management [33] IEEE Transactions on Signal Processing [34] IEEE Transactions on Vehicular Technology [35] IEEE Transactions on Wireless Communications [36,37] International Journal of Network Management [38] Performance Evaluation [39] Sensors [40] Transactions on Emerging Telecommunications Technologies [41] conducting a thorough literature search on the graph-based models. For now, we would give a short introduction for the GNNs used in the surveyed studies.…”

Section: Journal Name Studiesmentioning

confidence: 99%

“…How- Network Slicing [61] DQN with GAT GAT [89] Power Control [60] Heterogeneous GNNs HetGAT [96] Routing [17] GCLR MPNN [90] Traffic Prediction [30] Graph-based TCN GraphSAGE [84] Traffic Prediction [50,31] GASTN S-RNN [95] Traffic Prediction [15] DC-STGCN GCN [88] User Association, Power Control [62] Unsupervised Graph Model GraphSAGE [84] VNE [73,46] DRL with GCN GCN [88] ever, there are still many challenges for this objective to happen. To minimize the content fetching delay in D2D network, the joint optimization of cooperative caching and fetching is considered in [35] and a DRL-based algorithm is proposed. In the proposed algorithm, GAT is used for cooperative inter-agent coordination.…”

Section: Other Wireless Networkmentioning

confidence: 99%

See 1 more Smart Citation

Graph-based Deep Learning for Communication Networks: A Survey

Jiang

2021

Preprint

View full text Add to dashboard Cite

Communication networks are important infrastructures in contemporary society. There are still many challenges that are not fully solved and new solutions are proposed continuously in this active research area. In recent years, to model the network topology, graph-based deep learning has achieved state-of-the-art performance in a series of problems in communication networks. In this survey, we review the rapidly growing body of research using different graph-based deep learning models, e.g. graph convolutional and graph attention networks, in various problems from different communication networks, e.g. wireless networks, wired networks, and software-defined networks. We also present a well-organized list of the problem and solution for each study and identify future research directions. To the best of our knowledge, this paper is the first survey that focuses on the application of graph-based deep learning methods in communication networks. To track the follow-up research, a public GitHub repository is created, where the relevant papers will be updated continuously.

show abstract

Section: Journal Name Studiesmentioning

confidence: 99%

Section: Other Wireless Networkmentioning

confidence: 99%

Graph-based Deep Learning for Communication Networks: A Survey

Jiang

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, the authors in [58] model the spectrum access issue as a Dec-POMDP, where each V2V link as an agent selects a spectrum resource only depending on its local Channel State Information (CSI) to maximize the long term sum-rate of all the links since the V2V link may not get the instantaneous global CSI. The cooperative content caching problem in a D2D scenario investigated in [59] is modeled as a Dec-POMDP due to the fact that each D2D user as an agent cannot access the content caching status of all the users.…”

Section: B Dec-pomdpmentioning

confidence: 99%

“…Besides, the proposed JAL based CUCB can increase the cache hit rate of the IL based CUCB and LRU by 40% and 25%, respectively. The same D2D cooperative caching caching problem is also investigated in [59], where the problem is formulated as a Dec-POMDP to minimize the total content fetching latency. The authors extend the soft actor-critic (SAC) [67] to multi-agent environment as multi-agent SAC (MASAC) to solve the problem.…”

Section: B Content Cachingmentioning

confidence: 99%

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Li¹,

Zhu²,

Luong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues. Finally, we discuss the challenges, open issues, and future directions related to the applications of MARL in future Internet.

show abstract

“…Caching in vehicles and receiving the content through device-to-device (D2D) communication reduces the burden of backhaul without installing additional infrastructure. However, compared to the RSU, the communicable distance of vehicles is shorter, and the storage space of vehicles is smaller [ 6 ]. If both RSUs and vehicles are used to cache content, their problems would be complementary to each other.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Edge Caching with Mobility Prediction in Vehicular Networks

Choi

Lim

2023

Sensors

View full text Add to dashboard Cite

As vehicles are connected to the Internet, various services can be provided to users. However, if the requests of vehicle users are concentrated on the remote server, the transmission delay increases, and there is a high possibility that the delay constraint cannot be satisfied. To solve this problem, caching can be performed at a closer proximity to the user which in turn would reduce the latency by distributing requests. The road side unit (RSU) and vehicle can serve as caching nodes by providing storage space closer to users through a mobile edge computing (MEC) server and an on-board unit (OBU), respectively. In this paper, we propose a caching strategy for both RSUs and vehicles with the goal of maximizing the caching node throughput. The vehicles move at a greater speed; thus, if positions of the vehicles are predictable in advance, this helps to determine the location and type of content that has to be cached. By using the temporal and spatial characteristics of vehicles, we adopted a long short-term memory (LSTM) to predict the locations of the vehicles. To respond to time-varying content popularity, a deep deterministic policy gradient (DDPG) was used to determine the size of each piece of content to be stored in the caching nodes. Experiments in various environments have proven that the proposed algorithm performs better when compared to other caching methods in terms of the throughput of caching nodes, delay constraint satisfaction, and update cost.

show abstract

Cooperative Caching and Fetching in D2D Communications - A Fully Decentralized Multi-Agent Reinforcement Learning Approach

Cited by 28 publications

References 33 publications

Graph-based Deep Learning for Communication Networks: A Survey

Graph-based Deep Learning for Communication Networks: A Survey

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Deep Reinforcement Learning for Edge Caching with Mobility Prediction in Vehicular Networks

Contact Info

Product

Resources

About