A Multi-Agent Reinforcement Learning Approach for Conflict Resolution in Dense Traffic Scenarios

Lai, Jiajian; Cai, Kai-Yuan; Liu, Zhaoxuan; Yang, Yang

doi:10.1109/dasc52595.2021.9594437

Cited by 5 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The methods of Pham et al (2022), Wang et al (2019), andLai et al (2021) employ the approach of selecting one or multiple rerouting points to deconflict. To adapt to the current airspace operation mode based on flight level, a natural extension would be to modify the action from (𝑥, 𝑦) to (𝑥, 𝑦, 𝑘), where 𝑥 and 𝑦 denote the coordinates of the rerouting point, and 𝑘 represents the flight level of the rerouting point, with options like 0, 1, or 2, indicating that the rerouting point is at the current flight level, one level below, or one level above, respectively.…”

Section: Conflict Resolution Methods Based On Reinforcement Learningmentioning

confidence: 99%

A General Real-Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi-Agent Reinforcement Learning

CHEN

Yang

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Conflict Resolution Methods Based On Reinforcement Learningmentioning

confidence: 99%

A General Real-Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi-Agent Reinforcement Learning

CHEN

Yang

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Owing to the wide use-cases of UAVs, controlling them autonomously using artificial intelligence (AI) techniques has been studied extensively [46]- [49]. Among them, reinforcement learning is used to learn the optimal behavior in each situation through a reward.…”

Section: Introductionmentioning

confidence: 99%

Indoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning

Park

Jang

Shin

2021

2021 21st International Conference on Control, Automation and Systems (ICCAS)

View full text Add to dashboard Cite

Multi-agent reinforcement learning was performed in this study for indoor path planning of two unmanned aerial vehicles (UAVs). Each UAV performed the task of moving as fast as possible from a randomly paired initial position to a goal position in an environment with obstacles. To minimize training time and prevent the damage of UAVs, learning was performed by simulation. Considering the non-stationary characteristics of the multi-agent environment wherein the optimal behavior varies based on the actions of other agents, the action of the other UAV was also included in the state space of each UAV. Curriculum learning was performed in two stages to increase learning efficiency. A goal rate of 89.0% was obtained compared with other learning strategies that obtained goal rates of 73.6% and 79.9%.

show abstract

Priority-based intelligent resolution method of multi-aircraft flight conflicts

Sui,

Zhou,

Cui

2024

Aeronaut. j.

View full text Add to dashboard Cite

The rising demand for air traffic will inevitably result in a surge in both the number and complexity of flight conflicts, necessitating intelligent strategies for conflict resolution. This study addresses the critical challenges of scalability and real-time performance in multi-aircraft flight conflict resolution by proposing a comprehensive method that integrates a priority ranking mechanism with a conflict resolution model based on the Markov decision process (MDP). Within this framework, the proximity between aircraft in a multi-aircraft conflict set is dynamically assessed to establish a conflict resolution ranking mechanism. The problem of multi-aircraft conflict resolution is formalised through the MDP, encompassing the design of state space, discrete action space and reward function, with the transition function implemented via simulation prediction using model-free methods. To address the positional uncertainty of aircraft in real-time scenarios, the conflict detection mechanism introduces the aircraft’s positional error. A deep reinforcement learning (DRL) environment is constructed incorporating actual airspace structures and traffic densities, leveraging the Actor Critic using Kronecker-factored Trust Region (ACKTR) algorithm to determine resolution actions. The experimental results indicate that with 20–30 aircraft in the airspace, the success rate can reach 94% for the training set and 85% for the test set. Furthermore, this study analyses the impact of varying aircraft numbers on the success rate within a specific airspace scenario. The outcomes of this research provide valuable insights for the automation of flight conflict resolution.

show abstract

A Multi-Agent Reinforcement Learning Approach for Conflict Resolution in Dense Traffic Scenarios

Cited by 5 publications

References 11 publications

A General Real-Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi-Agent Reinforcement Learning

A General Real-Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi-Agent Reinforcement Learning

Indoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning

Priority-based intelligent resolution method of multi-aircraft flight conflicts

Contact Info

Product

Resources

About