Multi-Agent Reinforcement Learning in NOMA-Aided UAV Networks for Cellular Offloading

Zhong, Ruikang; Liu, Xiao; Liu, Yuanwei; Chen, Yue

doi:10.1109/twc.2021.3104633

Cited by 53 publications

(13 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the work [124], the authors suggested multi-agent reinforcement learning (MARL) algorithms, in which two UAVs decide the goal helper and bandwidth allocation, to separate the assessment of the integrated decision. Similarly, to maximize the long-term utility of the proposed UAV-enabled MEC network, the aim of the optimization problem in [125] is to find the best computation offloading and resource management policies. The problem is further formulated as a semi-Markov method (SMDP), and deep RL based algorithms are proposed in both centralized and distributed UAV-enabled MEC networks, taking into account random system demands and time-varying communication channels.…”

Section: Markov Decision Process and Reinforcement Learningmentioning

confidence: 99%

A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions

Baktayan¹,

Al-Baltah²

2022

Sustainable Engineering and Innovation

View full text Add to dashboard Cite

The lack of resource constraints for edge servers makes it difficult to simultaneously perform a large number of Mobile Devices’ (MDs) requests. The Mobile Network Operator (MNO) must then select how to delegate MD queries to its Mobile Edge Computing (MEC) server in order to maximize the overall benefit of admitted requests with varying latency needs. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligent (AI) can increase MNO performance because of their flexibility in deployment, high mobility of UAV, and efficiency of AI algorithms. There is a trade-off between the cost incurred by the MD and the profit received by the MNO. Intelligent computing offloading to UAV-enabled MEC, on the other hand, is a promising way to bridge the gap between MDs' limited processing resources, as well as the intelligent algorithms that are utilized for computation offloading in the UAV-MEC network and the high computing demands of upcoming applications. This study looks at some of the research on the benefits of computation offloading process in the UAV-MEC network, as well as the intelligent models that are utilized for computation offloading in the UAV-MEC network. In addition, this article examines several intelligent pricing techniques in different structures in the UAV-MEC network. Finally, this work highlights some important open research issues and future research directions of Artificial Intelligent (AI) in computation offloading and applying intelligent pricing strategies in the UAV-MEC network.

show abstract

Section: Markov Decision Process and Reinforcement Learningmentioning

confidence: 99%

A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions

Baktayan¹,

Al-Baltah²

2022

Sustainable Engineering and Innovation

View full text Add to dashboard Cite

show abstract

“…The authors of [202] jointly optimized the three-dimensional trajectory of multiple UAVs and the power allocation policy to maximize the total throughput in a UAV-BS enabled NOMA network. A Mutual Deep Q-network (MDQN) algorithm was proposed to solve the formulated problem.…”

Section: Bnbmentioning

confidence: 99%

“…Unsupervised learning algorithms, such as the principal component analysis (PCA) algorithm [214] and the K-means algorithm [215]- [217], have been used for user clustering in NOMA networks. NOMA S-IoT DL-PA Network utility Long-term power allocation [188] MIMO-NOMA CDNN EE Several convolutional layers and multiple hidden layers [189], [190] Multi-user NOMA DNN Sum rate Imperfect successive interference cancellation [191] NOMA-D2D DNN Sum rate Channel coefficients, power budget, and binary user pairing variable serve as DNN input [192] Multi-user NOMA DNN Transmission delay Dynamic power control [193] NOMA-SWIPT DBN Rate and harvested energy Comprises three phases: Preparing data samples, training, and running [197] Multi-user NOMA Hotbooting Q-learning Sum rate Without relying on the knowledge of the jamming and radio channel parameters [198] Grant-free NOMA LSTM-based DQN Throughput The long-term cluster throughput maximization problem is formulated as a POMDP problem [199] Hybrid NOMA Actor-critic Sum rate The proposed approach includes recurrent neural network (RNN) and proximal policy optimization (PPO) [200] MC-NOMA DDPG-based DRL-JRM Weighted sum rate A novel centralized action-value function is designed to measure the reward [201] NOMA-UAV CDRL Network capacity The proposed algorithm is based on PPO algorithm [202] NOMA-UAV MDQN Throughput…”

Section: ) Other Machine Learning Techniques For Nomamentioning

confidence: 99%

Evolution of NOMA Toward Next Generation Multiple Access (NGMA) for 6G

Liu,

Zhang,

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the explosive growth in the number of wireless devices and diverse wireless services, such as virtual/augmented reality and Internet-of-Everything, next generation wireless networks face unprecedented challenges caused by heterogeneous data traffic, massive connectivity, and ultra-high bandwidth efficiency and ultra-low latency requirements. To address these challenges, advanced multiple access schemes are expected to be developed, namely next generation multiple access (NGMA), which are capable of supporting massive numbers of users in a more resource-and complexity-efficient manner than existing multiple access schemes. As the research on NGMA is in a very early stage, in this paper, we explore the evolution of NGMA with a particular focus on non-orthogonal multiple access (NOMA), i.e., the transition from NOMA to NGMA. In particular, we first review the fundamental capacity limits of NOMA, elaborate the new requirements for NGMA, and discuss several possible candidate techniques. Moreover, given the high compatibility and flexibility of NOMA, we provide an overview of current research efforts on multi-antenna techniques for NOMA, promising future application scenarios of NOMA, and the interplay between NOMA and other emerging physical layer techniques. Furthermore, we discuss advanced mathematical tools for facilitating the design of NOMA communication systems, including conventional optimization approaches and new machine learning techniques. Next, we propose

show abstract

“…The previous works related to our paper include those focusing on wireless powered MEC networks [1]- [5], [10]- [14], and UAV-assisted communication networks for mobile terminals [15]- [17].…”

Section: Related Workmentioning

confidence: 99%

“…Ref. [17] discretized the flight direction of UAV and the transmit power of terminals and devised a value-based DRL algorithm. Since this algorithm has to search the action space exhaustively in each iteration, it cannot be used for problems with high-dimensional or continuous actions [18].…”

Section: B Uav-assisted Communication Network For Mobile Terminalsmentioning

confidence: 99%

Computation Rate Maximum for Mobile Terminals in UAV-assisted Wireless Powered MEC Networks with Fairness Constraint

Zhou¹,

Huang²,

Ye³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper investigates an unmanned aerial vehicle (UAV)-assisted wireless powered mobile-edge computing (MEC) system, where the UAV powers the mobile terminals by wireless power transfer (WPT) and provides computation service for them. We aim to maximize the computation rate of terminals while ensuring fairness among them. Considering the random trajectories of mobile terminals, we propose a soft actor-critic (SAC)-based UAV trajectory planning and resource allocation (SAC-TR) algorithm, which combines off-policy and maximum entropy reinforcement learning to promote the convergence of the algorithm. We design the reward as a heterogeneous function of computation rate, fairness, and reaching of destination. Simulation results show that SAC-TR can quickly adapt to varying network environments and outperform representative benchmarks in a variety of situations.

show abstract

Multi-Agent Reinforcement Learning in NOMA-Aided UAV Networks for Cellular Offloading

Cited by 53 publications

References 53 publications

A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions

A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions

Evolution of NOMA Toward Next Generation Multiple Access (NGMA) for 6G

Computation Rate Maximum for Mobile Terminals in UAV-assisted Wireless Powered MEC Networks with Fairness Constraint

Contact Info

Product

Resources

About