Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework

Zhan, Guang; Zhang, Xinmiao; Li, Zhongchao; Xu, Lin; Zhou, Dianwu; Yang, Zhen

doi:10.3390/drones6070166

Cited by 26 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Planning a multi-agent system is even more challenging since the planning space increases exponentially with the number of agents [24]. Due to the exponential growth of the size of the planning space with respect to the number of agents, scalability is recognized as a crucial limitation for the application of multi-agent planning algorithms to real-world scenarios [25]. Instead of using standard approaches such as domain-knowledge-supported algorithms that require human-level intervention in the planning phase, more popular techniques such as deep learning will be utilized.…”

Section: Significance Of Scalabilitymentioning

confidence: 99%

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Tarhan,

Ure

2024

Drones

View full text Add to dashboard Cite

The popularity of commercial unmanned aerial vehicles has drawn great attention from the e-commerce industry due to their suitability for last-mile delivery. However, the organization of multiple aerial vehicles efficiently for delivery within limitations and uncertainties is still a problem. The main challenge of planning is scalability, since the planning space grows exponentially to the number of agents, and it is not efficient to let human-level supervisors structure the problem for large-scale settings. Algorithms based on Deep Q-Networks had unprecedented success in solving decision-making problems. Extension of these algorithms to multi-agent problems is limited due to scalability issues. This work proposes an approach that improves the performance of Deep Q-Networks on multi-agent delivery by drone problems by utilizing state decompositions for lowering the problem complexity, Curriculum Learning for handling the exploration complexity, and Genetic Algorithms for searching efficient packet-drone matching across the combinatorial solution space. The performance of the proposed method is shown in a multi-agent delivery by drone problem that has 10 agents and ≈1077 state–action pairs. Comparative simulation results are provided to demonstrate the merit of the proposed method. The proposed Genetic-Algorithm-aided multi-agent DRL outperformed the rest in terms of scalability and convergent behavior.

show abstract

Section: Significance Of Scalabilitymentioning

confidence: 99%

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Tarhan,

Ure

2024

Drones

View full text Add to dashboard Cite

show abstract

“…The samples were distributed in proportion to the data in the original dataset [9]. The next step was to find and slightly modify the architecturally constructed neural network used in studies on detecting the presence and types of landmines [10]. The selected neural network consists of four layers of twodimensional convolution, five layers of one-dimensional maximum pooling, and nine layers of three-dimensional convolution.…”

Section: Detecting Landmines and Minefields Using Ai-mentioning

confidence: 99%

Інтеграція БпЛА Зі Нейронною Мережею У Завданнях Із Виявлення Наземних Мін Та Мінних Полів

Kasianchuk,

Lastivka

2023

SISIOT

View full text Add to dashboard Cite

One of the newest stages in the improvement of unmanned aerial vehicles (UAVs) is the integration of such systems with the neural networks, which, in turn, is not a novelty, but provides such systems with a further level of practical application. Having conducted a meta-analysis of the results of previous studies and available information on this topic, it was found that in the modern period, in addition to successful practical implementations of the integration of artificial intelligence with UAVs, there is already a certain classification of such processes according to the principles of optimal improvement of UAV capabilities and by areas of society. In addition to the publicly available and well-known information about the successful use of drones in the military and logistics sectors of human activity, UAVs successfully perform tasks in such sectors as agriculture, engineering, search, etc. The main purpose of the article is to analyze, review, study and systematize existing information on the positive effectiveness and feasibility of using the principles, approaches and integration of unmanned aerial vehicles with machine learning technologies to improve the efficiency of solving the problems of locating and detecting landmines and minefields, which is a major humanitarian problem for civil society located in the territory where military conflicts are currently taking place or in the territories where military clashes or conflicts have occurred in the past. In this article, a small study was conducted to develop a prototype neural network that can be further integrated with UAVs for landmine and minefield detection tasks. The described neural network was trained on an open dataset, trained using the algorithms chosen in the study, and has a fairly good final result in terms of detection accuracy, which is 1.5% higher than the accuracy of publicly available neural networks in a review of similar developments or studies.

show abstract

“…These training methods can train a new model of complex tasks based on the model trained from simple tasks and accelerate the convergence speed. An inheritance training method [ 16 ] based on the multi-agent proximal policy optimization method is developed to improve the generalization performance of the model. The idea of course learning is adopted in the method, and the results show that UAVs can search for and attack targets outside the training area.…”

Section: Introductionmentioning

confidence: 99%

A Bio-Inspired Decision-Making Method of UAV Swarm for Attack-Defense Confrontation via Multi-Agent Reinforcement Learning

Chi

Wei

et al. 2023

Biomimetics

View full text Add to dashboard Cite

The unmanned aerial vehicle (UAV) swarm is regarded as having a significant role in modern warfare. The demand for UAV swarms with the capability of attack-defense confrontation is urgent. The existing decision-making methods of UAV swarm confrontation, such as multi-agent reinforcement learning (MARL), suffer from an exponential increase in training time as the size of the swarm increases. Inspired by group hunting behavior in nature, this paper presents a new bio-inspired decision-making method for UAV swarms for attack-defense confrontation via MARL. Firstly, a UAV swarm decision-making framework for confrontation based on grouping mechanisms is established. Secondly, a bio-inspired action space is designed, and a dense reward is added to the reward function to accelerate the convergence speed of training. Finally, numerical experiments are conducted to evaluate the performance of our method. The experiment results show that the proposed method can be applied to a swarm of 12 UAVs, and when the maximum acceleration of the enemy UAV is within 2.5 times ours, the swarm can well intercept the enemy, and the success rate is above 91%.

show abstract

Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework

Cited by 26 publications

References 21 publications

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Інтеграція БпЛА Зі Нейронною Мережею У Завданнях Із Виявлення Наземних Мін Та Мінних Полів

A Bio-Inspired Decision-Making Method of UAV Swarm for Attack-Defense Confrontation via Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About