Toward Conflict Resolution with Deep Multi-Agent Reinforcement Learning

Isufaj, Ralvi; Sebastia, David Aranega; Piera, Miquel Àngel

doi:10.2514/1.d0296

Cited by 13 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such algorithms can be used to improve the quality of resolutions, in addition to solving the present conflicts. For instance, Isufaj et al [35] propose a multi-agent reinforcement learning approach to conflict resolution which considers airspace complexity as one of the factors that the model must optimize in addition to solving conflicts. The indicators proposed in this work could allow for a more granular optimization of complexity by providing more detailed information.…”

Section: Discussionmentioning

confidence: 99%

Spatiotemporal Graph Indicators for Air Traffic Complexity Analysis

2021

View full text Add to dashboard Cite

There has been extensive research in formalising air traffic complexity, but existing works focus mainly on a metric to tie down the peak air traffic controllers workload rather than a dynamic approach to complexity that could guide both strategical, pre-tactical and tactical actions for a smooth flow of aircraft. In this paper, aircraft interdependencies are formalized using graph theory and four complexity indicators are described, which combine spatiotemporal topological information with the severity of the interdependencies. These indicators can be used to predict the dynamic evolution of complexity, by not giving one single score, but measuring complexity in a time window. Results show that these indicators can capture complex spatiotemporal areas in a sector and give a detailed and nuanced view of sector complexity.

show abstract

Section: Discussionmentioning

confidence: 99%

Spatiotemporal Graph Indicators for Air Traffic Complexity Analysis

2021

View full text Add to dashboard Cite

show abstract

“…Furthermore, recent works saw the engagement of DRL methods, which behave better in multiagent environments and consider uncertainties. In this paper [25], the authors model pairwise conflict resolutions as a multiagent reinforcement learning (MARL) problem. They use Multiagent Deep Deterministic Policy Gradient (MADDPG) [26] to train two agents, representing each aircraft in a conflict pair, capable of efficiently solving conflicts in the presence of surrounding traffic by considering heading and speed changes.…”

Section: Related Workmentioning

confidence: 99%

Multi-UAV Conflict Resolution with Graph Convolutional Reinforcement Learning

2022

View full text Add to dashboard Cite

Safety is the primary concern when it comes to air traffic. In-flight safety between Unmanned Aircraft Vehicles (UAVs) is ensured through pairwise separation minima, utilizing conflict detection and resolution methods. Existing methods mainly deal with pairwise conflicts, however, due to an expected increase in traffic density, encounters with more than two UAVs are likely to happen. In this paper, we model multi-UAV conflict resolution as a multiagent reinforcement learning problem. We implement an algorithm based on graph neural networks where cooperative agents can communicate to jointly generate resolution maneuvers. The model is evaluated in scenarios with 3 and 4 present agents. Results show that agents are able to successfully solve the multi-UAV conflicts through a cooperative strategy.

show abstract

“…In [15] the authors combine Kernel Based RL with deep MARL to resolve conflicts by applying speed changes in real-time, also considering other factors such as fuel consumption and airspace congestion. Authors in [26] use Multi-Agent Deep Deterministic Policy Gradient (MADPG) to resolve conflicts, also considering time, fuel consumption and airspace complexity. Closer to our approach are methods that somehow consider the ATCO preferences, either in a data-driven way as in [27], [28] and [29], or by using rules and procedures derived from human experts as in [30].…”

Section: Related Workmentioning

confidence: 99%

Data-driven prediction of Air Traffic Controllers reactions to resolving conflicts

Bastas¹,

Vouros²

2022

Preprint

View full text Add to dashboard Cite

With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the Air Traffic Management domain, in this paper we propose deep learning techniques (DL) that can learn models of Air Traffic Controllers' (ATCO) reactions in resolving conflicts that can violate separation minimum constraints among aircraft trajectories: This implies learning when the ATCO will react towards resolving a conflict, and how he/she will react. Timely reactions, to which this paper aims, focus on when do reactions happen, aiming to predict the trajectory points, as the trajectory evolves, that the ATCO issues a conflict resolution action, while also predicting the type of resolution action (if any). Towards this goal, the paper formulates the ATCO reactions prediction problem for CD&R, and presents DL methods that can model ATCO timely reactions and evaluates these methods in real-world data sets, showing their efficacy in prediction with very high accuracy.

show abstract

Toward Conflict Resolution with Deep Multi-Agent Reinforcement Learning

Cited by 13 publications

References 17 publications

Spatiotemporal Graph Indicators for Air Traffic Complexity Analysis

Spatiotemporal Graph Indicators for Air Traffic Complexity Analysis

Multi-UAV Conflict Resolution with Graph Convolutional Reinforcement Learning

Data-driven prediction of Air Traffic Controllers reactions to resolving conflicts

Contact Info

Product

Resources

About