Adaptive UAV Swarm Mission Planning by Temporal Difference Learning

Gopalakrishnan, Shreevanth Krishnaa; Al‐Rubaye, Saba; Tsourdos, Antonios

doi:10.1109/dasc52595.2021.9594300

Cited by 5 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, cloud computing must improve to cater for the high data management demands brought about by data-driven algorithms [48]. Cybersecurity protocols must also evolve to combat the cyber vulnerabilities of safety-critical ICPSs [49][50][51].…”

Section: Literature Gaps and Contributionsmentioning

confidence: 99%

Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems

2023

Self Cite

View full text Add to dashboard Cite

Intelligent vehicular cyber-physical systems (ICPSs) increase the reliability, efficiency and adaptability of urban mobility systems. Notably, ICPSs enable autonomous transportation in smart cities, exemplified by the emerging fields of self-driving cars and advanced air mobility. Nonetheless, the deployment of ICPSs raises legitimate concerns surrounding safety assurance, cybersecurity threats, communication reliability, and data management. Addressing these issues often necessitates specialised platforms to cater to the heterogeneity and complexity of ICPSs. To address this challenge, this paper presents a comprehensive CPS to explore, develop and test ICPSs and intelligent vehicular algorithms. A customisable embedded system is realised using a field programmable gate array, which is connected to a supervisory computer to enable networked operations and support advanced multi-agent algorithms. The platform remains compatible with multiple vehicular sensors, communication protocols and human–machine interfaces, essential for a vehicle to perceive its surroundings, communicate with collaborative systems, and interact with its occupants. The proposed CPS thereby offers a practical resource to advance ICPS development, comprehension, and experimentation in both educational and research settings. By bridging the gap between theory and practice, this tool empowers users to overcome the complexities of ICPSs and contribute to the emerging fields of autonomous transportation and intelligent vehicular systems.

show abstract

Section: Literature Gaps and Contributionsmentioning

confidence: 99%

Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems

2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several research studies have investigated the application of DRL in the field of communications in recent years, as evidenced by publications such as [8,[27][28][29]. In particular, ref.…”

Section: Background and Related Workmentioning

confidence: 99%

“…In addition, dynamic routes could also render this method ineffective. Even though current AI/ML approaches such as [8,[27][28][29][30] have been effective at mitigating interference, these methods have been deployed for UAVs in general or for users in 5G networks but not both, which is the unique problem that the proposed method in this paper solves. In Table 1, the literature is summarised and shows that the current solutions for mitigating interference are not sufficient.…”

Section: Outcomementioning

confidence: 99%

See 1 more Smart Citation

AI-Enabled Interference Mitigation for Autonomous Aerial Vehicles in Urban 5G Networks

Warrier,

Al-Rubaye,

Inalhan

et al. 2023

Aerospace

Self Cite

View full text Add to dashboard Cite

Integrating autonomous unmanned aerial vehicles (UAVs) with fifth-generation (5G) networks presents a significant challenge due to network interference. UAVs’ high altitude and propagation conditions increase vulnerability to interference from neighbouring 5G base stations (gNBs) in the downlink direction. This paper proposes a novel deep reinforcement learning algorithm, powered by AI, to address interference through power control. By formulating and solving a signal-to-interference-and-noise ratio (SINR) optimization problem using the deep Q-learning (DQL) algorithm, interference is effectively mitigated, and link performance is improved. Performance comparison with existing interference mitigation schemes, such as fixed power allocation (FPA), tabular Q-learning, particle swarm optimization, and game theory demonstrates the superiority of the DQL algorithm, where it outperforms the next best method by 41.66% and converges to an optimal solution faster. It is also observed that, at higher speeds, the UAV sees only a 10.52% decrease in performance, which means the algorithm is able to perform effectively at high speeds. The proposed solution effectively integrates UAVs with 5G networks, mitigates interference, and enhances link performance, offering a significant advancement in this field.

show abstract

“…In [13], the authors propose a new cooperative interference cancellation strategy for the multi-beam UAV uplink communication, which aims to eliminate the co-channel interference at each of the occupied BSs and maximize the sum-rate to the available BSs. Over the last few years, the use of deep learning in wireless communications was studied in certain literature [14]. Specifically, [15] uses deep reinforcement learning to perform power control for mmWave and this was designed as an alternative to beamforming in improving the non-line of sight (NLOS) transmission performance.…”

Section: Background and Related Workmentioning

confidence: 99%

Interference Mitigation for 5G-Connected UAV using Deep Q-Learning Framework

Warrier

Al‐Rubaye

Panagiotakopoulos

et al. 2022

2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC)

View full text Add to dashboard Cite

To boost large-scale deployment of unmanned aerial vehicles (UAVs) in the future, a new wireless communication paradigm namely cellular-connected UAVs has recently received an upsurge of interest in both academia and industry. Fifth generation (5G) networks are expected to support this largescale deployment with high reliability and low latency. Due to the high mobility, speed, and altitude of the UAVs there are numerous challenges that hinder its integration with the 5G architecture. Interference is one of the major roadblocks to ensuring the efficient co-existence between UAVs and terrestrial users in 5G networks. Conventional interference mitigation schemes for terrestrial networks are insufficient to deal with the more severe air-ground interference, which thus motivates this paper to propose a new algorithm to mitigate interference. A deep Q-learning (DQL) based algorithm is developed to mitigate interference intelligently through power control. The proposed algorithm formulates a non-convex optimization problem to maximize the Signal to Interference and Noise Ratio (SINR) and solves it using DQL. Its performance is measured as effective SINR against the complement cumulative distribution function. Further, it is compared with an adaptive link technique: Fixed Power Allocation (FPA), a standard power control scheme and tabular Q-learning(TQL). It is seen that the FPA has the worst performance while the TQL performs slightly better. This is since power control and interference coordination are introduced but not as effectively in the TQL method. It is observed that DQL algorithm outperforms the TQL implementation. To solve the severe air-ground interference experienced by the UAVs in 5G networks, this paper proposes a DQL algorithm. The algorithm effectively mitigates interference by optimizing SINR of the air-ground link and outperforms the existing methods. This paper therefore, proposes an effective algorithm to resolve the interference challenge in air-ground links for 5G-connected UAVs.

show abstract

Adaptive UAV Swarm Mission Planning by Temporal Difference Learning

Cited by 5 publications

References 27 publications

Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems

Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems

AI-Enabled Interference Mitigation for Autonomous Aerial Vehicles in Urban 5G Networks

Interference Mitigation for 5G-Connected UAV using Deep Q-Learning Framework

Contact Info

Product

Resources

About