A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation

Dai, Zhaojun; Zhang, Yan; Zhang, Wancheng; Luo, Xinran; He, Zhiyuan

doi:10.1109/tsipn.2022.3150911

Cited by 32 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [12], a resource allocation strategy for UAV networks was proposed, employing multi-agent cooperative environment learning to enhance system performance. The study of [13] integrated UAVs' advantages with Ultra-Reliable Low-Latency Communication (URLLC) systems, primarily aiming to boost the transmission rate of the backward link.…”

Section: B Resource Allocation In Uav Aided Communication Systemsmentioning

confidence: 99%

MARS: A DRL-Based Multi-Task Resource Scheduling Framework for UAV With IRS-Assisted Mobile Edge Computing System

Jiang,

Peng,

Wang

et al. 2023

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

This paper investigates a dynamic Mobile Edge Computing (MEC) system enhanced by Unmanned Aerial Vehicles (UAVs) and Intelligent Reflective Surfaces (IRSs). We introduce a scalable resource scheduling algorithm aimed at reducing energy consumption for all User Equipments (UEs) and UAVs within the MEC system, accommodating a varying number of UAVs. To address this challenge, we present a Multi-tAsk Resource Scheduling (MARS) framework that employs Deep Reinforcement Learning (DRL). Firstly, we present a novel Advantage Actor-Critic (A2C) structure with the statevalue critic and entropy-enhanced actor to reduce variance and enhance the policy search of DRL. Then, we present a multi-head agent with three different heads in which a classification head is applied to make offloading decisions and a regression head is presented to allocate computational resource, and a critic head is introduced to estimate the state value of the selected action. Next, we introduce a multi-task controller to adjust the agent to adapt to the varying number of UAVs by loading or unloading a part of the weights in the agent. Finally, a Light Wolf Search (LWS) is introduced as the action refinement to enhance the exploration in the dynamic action space. Numerical results demonstrate the feasibility and efficiency of the MARS framework.Index Terms-Mobile edge computing (MEC), intelligent reflecting surface (IRS), unmanned aerial vehicle (UAV), deep reinforcement learning (DRL), resource scheduling.

show abstract

Section: B Resource Allocation In Uav Aided Communication Systemsmentioning

confidence: 99%

MARS: A DRL-Based Multi-Task Resource Scheduling Framework for UAV With IRS-Assisted Mobile Edge Computing System

Jiang,

Peng,

Wang

et al. 2023

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

show abstract

“…Howoever, since deep learning algorithms are dependent on hyperparameters, applying analytical methodologies to guarantee the convergence of the proposed DQL-based method is difficult. This is a common challenge in the literature for analytically proving optimality and convergence [53]- [56]. Therefore, instead of convergence, we are presenting a theorem that shows the amount of work per iteration in Algorithm 1, namely Theorem 1.…”

Section: Complexity Analysis Of the Proposed Algorithmmentioning

confidence: 99%

Deep Q-Learning-Based Resource Allocation in NOMA Visible Light Communications

Hammadi

Bariah

Muhaidat

et al. 2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Visible light communication (VLC) has been introduced as a key enabler for high-data rate wireless services in future wireless communication networks. In addition to this, it was also demonstrated recently that non-orthogonal multiple access (NOMA) can further improve the spectral efficiency of multi-user VLC systems. In this context and owing to the significantly promising potential of artificial intelligence in wireless communications, the present contribution proposes a deep Q-learning (DQL) framework that aims to optimize the performance of an indoor NOMA-VLC downlink network. In particular, we formulate a joint power allocation and LED transmission angle tuning optimization problem, in order to maximize the average sum rate and the average energy efficiency. The obtained results demonstrate that our algorithm offers a noticeable performance enhancement into the NOMA-VLC systems in terms of average sum rate and average energy efficiency, while maintaining the minimum convergence time, particularly for higher number of users. Furthermore, considering a realistic downlink VLC network setup, the simulation results have shown that our algorithm outperforms the genetic algorithm (GA) and the differential evolution (DE) algorithm in terms of average sum rate, and offers considerably less run-time complexity.

show abstract

“…Another new resource allocation algorithm for UAV networks based on multi-agent collaborative environment learning is proposed in [22]. It aims to overcome the communication delay and enhance the network efficiency, caused by the centralized architecture.…”

Section: Resource-constrained Client/participant Selection In Fl Processmentioning

confidence: 99%

Multi-Constrained and Edge-Enabled Selection of UAV Participants in Federated Learning Process

et al. 2022

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) have gained increasing attention in boosting the performance of conventional networks due to their small size, high efficiency, low cost, and autonomously nature. The amalgamation of UAVs with both distributed/collaborative Deep Learning (DL) algorithms, such as Federated Learning (FL), and Blockchain technology have ushered in a new paradigm of Secure Multi-Access Edge Computing (S-MEC). Indeed, FL enables UAV devices to leverage their sensed data to build local DL models. The latter are then sent to a central node, e.g., S-MEC node, for aggregation, in order to generate a global DL model. Therefore, FL enables UAV devices to collaborate during several FL rounds in generating a learning model, while avoiding to share their local data, and thus ensuring UAVs’ privacy. However, UAV devices are usually limited in terms of resources such as battery, memory, and CPU. Some of the UAV devices may not be able to build a local learning models due to their resources capacity. Hence, there is a great need to select the adequate UAVs at each FL round, that are able to build a local DL model based on their resource capacities. In this paper, we design a novel and S-MEC-enabled framework that optimizes the selection of UAV participants at each FL training round, named FedSel. FedSel considers the available UAVs along with their resource capacities, in terms of energy, CPU, and memory, to determine which UAV device is able to participant in the FL process. Thus, we formulate the UAV selection problem as an Integer Linear Program, which considers the aforementioned constraints. We also prove that this problem is NP-hard, and suggest a Tabu Search (TS) metaheuristic-based approach to resolve it. Moreover, FedSel is built on top of blockchain technology, in order to ensure a secure selection of UAV participants, and hence building reliable FL-based models. Simulation results validate the efficiency of our FedSel scheme in balancing computational load among available UAVs and optimizing the UAV selection process.

show abstract

A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation

Cited by 32 publications

References 38 publications

MARS: A DRL-Based Multi-Task Resource Scheduling Framework for UAV With IRS-Assisted Mobile Edge Computing System

MARS: A DRL-Based Multi-Task Resource Scheduling Framework for UAV With IRS-Assisted Mobile Edge Computing System

Deep Q-Learning-Based Resource Allocation in NOMA Visible Light Communications

Multi-Constrained and Edge-Enabled Selection of UAV Participants in Federated Learning Process

Contact Info

Product

Resources

About