A Path Planning Algorithm for UAV Based on Improved Q-Learning

Yan, Chao; Xiang, Xiaojia

doi:10.1109/icras.2018.8443226

Cited by 59 publications

(18 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the works in [ 56 , 57 ] utilized deep reinforcement learning techniques, such as Q-learning, as a promising solution to solve the problem of real-time drone path planning in unknown dynamic environments. Alternatively, heuristics intelligent optimization algorithms have also been widely used in recent years to solve the local path planning optimization problems, such as graph-based algorithms [ 58 ], heuristic search algorithms [ 59 ], field-based algorithms [ 60 ], and intelligent optimization algorithms [ 61 ].…”

Section: Related Workmentioning

confidence: 99%

LoRa Communications as an Enabler for Internet of Drones towards Large-Scale Livestock Monitoring in Rural Farms

Behjati

Noh

Alobaidy

et al. 2021

Sensors

View full text Add to dashboard Cite

Currently, smart farming is considered an effective solution to enhance the productivity of farms; thereby, it has recently received broad interest from service providers to offer a wide range of applications, from pest identification to asset monitoring. Although the emergence of digital technologies, such as the Internet of Things (IoT) and low-power wide-area networks (LPWANs), has led to significant advances in the smart farming industry, farming operations still need more efficient solutions. On the other hand, the utilization of unmanned aerial vehicles (UAVs), also known as drones, is growing rapidly across many civil application domains. This paper aims to develop a farm monitoring system that incorporates UAV, LPWAN, and IoT technologies to transform the current farm management approach and aid farmers in obtaining actionable data from their farm operations. In this regard, an IoT-based water quality monitoring system was developed because water is an essential aspect in livestock development. Then, based on the Long-Range Wide-Area Network (LoRaWAN®) technology, a multi-channel LoRaWAN® gateway was developed and integrated into a vertical takeoff and landing drone to convey collected data from the sensors to the cloud for further analysis. In addition, to develop LoRaWAN®-based aerial communication, a series of measurements and simulations were performed under different configurations and scenarios. Finally, to enhance the efficiency of aerial-based data collection, the UAV path planning was optimized. Measurement results showed that the maximum achievable LoRa coverage when operating on-air via the drone is about 10 km, and the Longley–Rice irregular terrain model provides the most suitable path loss model for the scenario of large-scale farms, and a multi-channel gateway with a spreading factor of 12 provides the most reliable communication link at a high drone speed (up to 95 km/h). Simulation results showed that the developed system can overcome the coverage limitation of LoRaWAN® and it can establish a reliable communication link over large-scale wireless sensor networks. In addition, it was shown that by optimizing flight paths, aerial data collection could be performed in a much shorter time than industrial mission planning (up to four times in our case).

show abstract

Section: Related Workmentioning

confidence: 99%

LoRa Communications as an Enabler for Internet of Drones towards Large-Scale Livestock Monitoring in Rural Farms

Behjati

Noh

Alobaidy

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Most of the developed models are limited to simplified 2D navigation space (i.e., fixed altitude), where the UAV is not able to change its altitude to cross over obstacles. In [32] and [33], the authors presented a Q-learning algorithm to solve the autonomous scheduling problem of UAVs. Q-learning was also employed to establish paths while avoiding obstacles in [34].…”

Section: A Literature Reviewmentioning

confidence: 99%

A UAV-Assisted Data Collection for Wireless Sensor Networks: Autonomous Navigation and Scheduling

et al. 2020

View full text Add to dashboard Cite

Nowadays, Wireless Sensor Networks (WSNs) are playing a vital and sustainable role in many verticals touching different aspects of our lives including civil, public, and military applications. WSNs majorly consist of a few to several sensor nodes, that are connected to each other via wireless communication links and require real-time or delayed data transfer. In this paper, we propose an autonomous Unmanned Aerial Vehicle (UAV)-enabled data gathering mechanism for delay-tolerant WSN applications. The objective is to employ a self-trained UAV as a flying mobile unit collecting data from ground sensor nodes spatially distributed in a given geographical area during a predefined period of time. In this approach, two Reinforcement Learning (RL) approaches, specifically Deep Deterministic Gradient Decent (DDPG) and Q-learning (QL) algorithms, are jointly employed to train the UAV to understand the environment and provide effective scheduling to accomplish its data collection mission. The DDPG is used to autonomously decide the best trajectory to adopt in an obstacle-constrained environment, while the QL is developed to determine the order of nodes to visit such that the data collection time is minimized. The schedule is obtained while considering the limited battery capacity of the flying unit, its need to return the charging station, the time windows of data acquisition, and the priority of certain sensor nodes. Customized reward functions are designed for each RL model and, through numerical simulations, we investigate their training performances. We also analyze the behavior of the autonomous UAV for different selected scenarios and corroborate the ability of the proposed approach in performing effective data collection. A comparison with the deterministic optimal solution is provided to validate the performance of the learning-based approach. INDEX TERMS Internet-of-things, data gathering, reinforcement learning, scheduling, unmanned areal vehicles

show abstract

“…Reinforcement learning (RL) is an artificial intelligence technique with relevant applications in robotics [8,15,[28][29][30]37], path planning [20,39,47,59,75,76] and combinatorial optimization problems [4,7,13,14,21,44,53,54,64,79], such as the TSP [1,2,18,22,41,45,52,66,81]. In RL, an agent learns from rewards and penalties in interacting with an environment [68].…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning for the traveling salesman problem with refueling

Ottoni

Nepomuceno

Oliveira

et al. 2021

Complex Intell. Syst.

View full text Add to dashboard Cite

The traveling salesman problem (TSP) is one of the best-known combinatorial optimization problems. Many methods derived from TSP have been applied to study autonomous vehicle route planning with fuel constraints. Nevertheless, less attention has been paid to reinforcement learning (RL) as a potential method to solve refueling problems. This paper employs RL to solve the traveling salesman problem With refueling (TSPWR). The technique proposes a model (actions, states, reinforcements) and RL-TSPWR algorithm. Focus is given on the analysis of RL parameters and on the refueling influence in route learning optimization of fuel cost. Two RL algorithms: Q-learning and SARSA are compared. In addition, RL parameter estimation is performed by Response Surface Methodology, Analysis of Variance and Tukey Test. The proposed method achieves the best solution in 15 out of 16 case studies.

show abstract

A Path Planning Algorithm for UAV Based on Improved Q-Learning

Cited by 59 publications

References 3 publications

LoRa Communications as an Enabler for Internet of Drones towards Large-Scale Livestock Monitoring in Rural Farms

LoRa Communications as an Enabler for Internet of Drones towards Large-Scale Livestock Monitoring in Rural Farms

A UAV-Assisted Data Collection for Wireless Sensor Networks: Autonomous Navigation and Scheduling

Reinforcement learning for the traveling salesman problem with refueling

Contact Info

Product

Resources

About