UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning

Theile, Mirco; Bayerlein, Harald; Nai, Richard; Gesbert, David; Caccamo, Marco

doi:10.1109/icar53236.2021.9659413

Cited by 29 publications

(20 citation statements)

References 23 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Boolean exclusion expressions (AND NOT) aim at disregarding artefacts which are too strictly applied to specific domains, with findings which are not necessarily transferable to the scope of our work. This is relevant mainly when considering general RL-based methods for balancing global and local effects, where several recent publications have emerged for Robotics path planning (Theile et al, 2021). cc 121 and 201 different artefacts were retrieved from q1 and q2, respectively.…”

Section: • Query 2 (Q2): Title-abs-key ( ( "Reinforcement Learning" )...mentioning

confidence: 99%

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Sousa

Henriques

2023

Preprint

View full text Add to dashboard Cite

In the current context of big data, the operational forecasting problems are more and more frequently involving the prediction of collections of multivariate, high-dimensional, related time series. The challenge of forecasting groups of time series can be tackled by fitting a single function to all series (global approach) or assuming each series to be a separate prediction problem and fitting one function to each problem (local approach). Although some global models show promising results against local benchmarks, there is still unclarity on how to best control generalization and structural assumptions by calibrating the exploitation of global patterns with local components. This is heavily reliant on the level of complexity from the dependency relationships exhibited between the series within a collection (e.g. indirect latent, local covariate relationships or noise effects) and other dimensions, such as the heterogeneity of the series and its length. For these reasons, there is an acknowledged need to invest further in developing scalable and data-driven hybrid models. The literature proposes Deep Learning (DL) based models built from different data generating processes (such as Auto-Regressive) which combine global models with classical local models (global-local frameworks). These methods frequently capture globality through factorization models or latent deep components and locality via classical local models or networks. These methods present non-linear modelling capabilities but are still either limited by solid theoretical assumptions (e.g. linearity of the underlying data) or predominantly one-dimensional, i.e., the future predictions for a single dimension mainly depend on past values from that same dimension. Simultaneously, dynamic Reinforcement Learning (RL) based models have been widely explored to optimize the balance of global and local signals in general regression problems, frequently through Q-learning algorithms. These derive from the equilibrium of cooperative strategies or the societal value in multi-agent learning systems. Similar models have been successfully applied to time series forecasting problems (such as stock market predictions), addressing the risk of method generalization under varying conditions. This paper conducted a concise literature review focused on these two research streams (global-local frameworks and RL based models) to optimize the balance between globality and locality in forecasting collections of time series. It focuses on their evolution across time and hints at opportunities to close some of the research gaps by intersecting both propositions. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and achieved a selection of 118 publications since 2000. The main findings revealed that global models have achieved strong expressiveness in capturing the most complex structural patterns while still enabling probabilistic outcomes to be delivered through uncertainty estimates. On the other hand, RL based methods depict great benefits in mitigating the risks of generalization by imprinting contextual diversity when predicting each step ahead for each series. Within those, the adoption of other computational learning or evolutionary-based methods (e.g. Genetic Programming) to improve the parametrization of the learning policies is also highlighted as an area of future work yet to be uncovered.

show abstract

Section: • Query 2 (Q2): Title-abs-key ( ( "Reinforcement Learning" )...mentioning

confidence: 99%

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Sousa

Henriques

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Di Franco and Buttazzo 116 proposed the technique to cover the whole searchable area with minimum energy consumption, but satisfying the coverage and time efficiency requirements. Theile et al 117 presented an approach for covering the whole area. Their method is based on DRL.…”

Section: Characteristics Of Coverage Path Planning For Uavmentioning

confidence: 99%

Classical versus reinforcement learning algorithms for unmanned aerial vehicle network communication and coverage path planning: A systematic literature review

Mannan

Obaidat

Mahmood

et al. 2022

Int J Communication

View full text Add to dashboard Cite

The unmanned aerial vehicle network communication includes all points of interest during the coverage path planning. Coverage path planning in such networks is crucial for many applications, such as surveying, monitoring, and disaster management. Since the coverage path planning belongs to NP-hard issues, researchers in this domain are constantly looking for optimal solutions for this task. The speed, direction, altitude, environmental variations, and obstacles make coverage path planning more difficult. Researchers have proposed numerous algorithms regarding coverage path planning. In this study, we examined and discussed existing state-of-the-art coverage path planning algorithms. We divided the existing techniques into two core categories: Classical and reinforcement learning. The classical algorithms are further divided into subcategories due to the availability of considerable variations in this category. For each algorithm in both types, we examined the issues of mobility, altitude, and characteristics of known and unknown environments. We also discuss the optimality of different algorithms. At the end of each section, we discuss the existing research gaps and provide future insights to overcome those gaps.

show abstract

“…In [4], the UAV constructs REM through the synergies between vision and communication in the edge network, which assists the UAV in the realization of online path planning and autonomous flight. The authors in [5] proposed a UAV path planning method, which exploits a compressed global map of the environment combined with a cropped but uncompressed local map showing the vicinity of the UAV. This method of distributing global and local map information for UAV path planning is inspired.…”

Section: Introductionmentioning

confidence: 99%

A 3D REM-Guided UAV Path Planning Method under Communication Connectivity Constraints

Liu

Zhou

Zhang

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the emergence of a large number of smart devices, the radio environment in which unmanned aerial vehicles (UAVs) take tasks is becoming more and more complex, which puts forward higher requirements for UAVs’ situational awareness and autonomous obstacle avoidance capabilities. To tackle this issue, we propose a three-dimension (3D) UAV path planning method under communication connectivity constraints guided by radio environment maps (REMs), which are distributed by ground edge servers in the form of compressed global REMs and detailed local REMs. An interfered fluid dynamic system (IFDS) model is deployed on UAVs to allow them to avoid obstacles and plan paths. We propose a twin-delayed deep deterministic policy gradient- (TD3-) based deep reinforcement learning (DRL) method to optimize the reaction coefficients of UAVs to avoid obstacles and improve the signal to interference plus noise ratio (SINR). The simulation results show that the proposed algorithm can effectively avoid static obstacles and dynamic interference under communication connectivity constraints, significantly improve the communication stability with a higher receive signal SINR and reduce the cost of UAV performing tasks with the shortest path.

show abstract

UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning

Cited by 29 publications

References 23 publications

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Classical versus reinforcement learning algorithms for unmanned aerial vehicle network communication and coverage path planning: A systematic literature review

A 3D REM-Guided UAV Path Planning Method under Communication Connectivity Constraints

Contact Info

Product

Resources

About