Multi-Agent RL based User-Centric Spectrum Allocation Scheme in D2D Enabled Hetnets

Zia, Kamran; Javed, Nauman; Sial, Muhammad Safdar; Ahmed, Sohail; Pervez, Farrukh

doi:10.1109/camad.2018.8514983

Cited by 11 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are some resource allocation studies based on multiagent RL [26], [27], [32], [33]. In [26], the resource allocation problem is modeled as a stochastic non-cooperative game and a Q-learning based algorithm is proposed.…”

Section: Arxiv:191209302v1 [Csni] 18 Dec 2019mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Guo

2020

IEEE Trans. Veh. Technol.

120

View full text Add to dashboard Cite

Device-to-device (D2D) communication underlay cellular networks is a promising technique to improve spectrum efficiency. In this situation, D2D transmission may cause severe interference to both the cellular and other D2D links, which imposes a great technical challenge to spectrum allocation. Existing centralized schemes require global information, which causes a large signaling overhead. While existing distributed schemes requires frequent information exchange among D2D users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multiagent deep reinforcement learning is proposed, named multiagent actor critic (MAAC). MAAC shares global historical states, actions and policies during centralized training, requires no signal interaction during execution and utilizes cooperation among users to further optimize system performance. Moreover, in order to decrease the computing complexity of the training, we further propose the neighbor-agent actor critic (NAAC) based on the neighbor users' historical information for centralized training. The simulation results show that the proposed MAAC and NAAC can effectively reduce the outage probability of cellular links, greatly improve the sum rate of D2D links and converge quickly.Index Terms-Device-to-device (D2D) communications, multiagent deep reinforcement learning, spectrum allocation.

show abstract

Section: Arxiv:191209302v1 [Csni] 18 Dec 2019mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Guo

2020

IEEE Trans. Veh. Technol.

120

View full text Add to dashboard Cite

show abstract

“…In this paper, we propose a distributive Reinforcement Learning (RL) based resource allocation scheme in multi-tier heterogeneous network to mitigate interference between D2D users and cellular users. In our previous work [19], we proposed a learning based autonomous spectrum allocation scheme for two-tier HetNets and shown the performance enhancement of D2D users in terms of throughput and spectral efficiency while meeting QoS requirements of conventional cellular users. In this paper, we have extended our work to multi-tier HetNets and analyzed the effects of increase in base station density and network tier over the performance of D2D users.…”

Section: B Motivationmentioning

confidence: 99%

A Distributed Multi-Agent RL-Based Autonomous Spectrum Allocation Scheme in D2D Enabled Multi-Tier HetNets

et al. 2019

Self Cite

View full text Add to dashboard Cite

Multi-tier heterogeneous networks (HetNets) and device-to-device (D2D) communication are vastly considered in 5G networks. The interference mitigation and resource allocation in the D2D enabled multi-tier HetNets is a cumbersome and challenging task that cannot be solved by the conventional centralized resource allocation techniques proposed in the literature. In this paper, we propose a distributed multi-agent learning-based spectrum allocation scheme in which D2D users learn the wireless environment and select spectrum resources autonomously to maximize their throughput and spectral efficiency (SE) while causing minimum interference to the cellular users. We have employed the distributed learning in a stochastic geometry-based realistic multi-tier heterogeneous network to validate the performance of our scheme. The proposed scheme enables the D2D users to achieve higher throughput and SE, higher signal-to-interferenceplus-noise ratio and low outage ratio for cellular users, and better computational time efficiency and performs well in the dense multi-tier HetNets without affecting network coverage compared with the distance based resource criterion and joint-resource allocation and link adaptation schemes.INDEX TERMS D2D communication, multi-agent reinforcement learning, autonomous spectrum allocation, distributed reinforcement learning, heterogeneous networks, interference mitigation in D2D enabled HetNets.

show abstract

“…Similarly, the interference can affect the D2D system if the same resource is shared between two DUs or DUs and CUs. To examine this, Zia et al 34 introduced a distributed learning algorithm for the spectrum allocation, where DU studied the environment and autonomously picked the resources to increase the system throughput and spectrum efficiency. Then, Kim 35 adopted a cooperative game theory to tackle the spectrum allocation problems.…”

Section: Introductionmentioning

confidence: 99%

A survey on energy‐efficient resource allocation schemes in device‐to‐device communication

Jadav

Gupta

Tanwar

2022

Int J Communication

View full text Add to dashboard Cite

Summary Device‐to‐device (D2D) communication is an emanating and robust technology for next‐generation wireless networks. It aims to minimize load from the centralized base station (BS) and lets the device communicate directly in the presence or absence of it. Further, to increase the spectral efficiency, it operates in an underlay mode, where D2D devices/users reuse the spectrum of cellular users (CUs), resulting in unusual intercell/intracell interferences. Managing resources efficiently can drastically reduce the effect of interference but on the cost of high energy consumption. The research community is diligently studying the optimal methods for energy‐efficient resource allocation in D2D systems. As per the literature, there exists no comprehensive survey that studied the existing energy‐efficient resource allocation schemes in the D2D environment. Motivated from this, we present a survey on recently adopted solutions for energy‐efficient power, channel, and spectrum allocation in D2D communication, which includes artificial intelligence (AI), optimization, dynamic programming, heuristic approach, game theory, and graph models. In addition, we also highlighted the open research issues and technical challenges while allocating resources to D2D users (DUs).

show abstract

Multi-Agent RL based User-Centric Spectrum Allocation Scheme in D2D Enabled Hetnets

Cited by 11 publications

References 10 publications

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

A Distributed Multi-Agent RL-Based Autonomous Spectrum Allocation Scheme in D2D Enabled Multi-Tier HetNets

A survey on energy‐efficient resource allocation schemes in device‐to‐device communication

Contact Info

Product

Resources

About