Drone Navigation and Avoidance of Obstacles Through Deep Reinforcement Learning

Abstract

Unmanned aerial vehicles (UAV) specifically drones have been used for surveillance, shipping and delivery, wildlife monitoring, disaster management etc. The increase on the number of drones in the airspace worldwide will lead necessarily to full autonomous drones. Given the expected huge number of drones, if they were operated by human pilots, the possibility to collide with each other could be too high.In this paper, deep reinforcement learning (DRL) architecture is proposed to make drones behave autonomously inside a suburb neighborhood environment. The environment in the simulator has plenty of obstacles such as trees, cables, parked cars and houses. In addition, there are also another drones, acting as moving obstacles, inside the environment while the other drone has a goal to achieve. In this way the drone can be trained to detect stationary and moving obstacles inside the neighborhood and so the drones can be used safely in a public area in the future. The drone has a front camera and it can capture continuously depth images. Every depth image, with a size of 144x256 pixels, is part of the state used in DRL architecture. Also, another part of the state is the distance to the geo-fence, a virtual barrier on the environment, which is added as a scalar value. The agent will be rewarded negatively when it tries to overpass the geo-fence limits. In addition, angle to goal and elevation angle between the goal and the drone will be used as information to be added to the state. It is considered that these scalar values will improve the DRL performance and also the reward obtained. The drone is trained using Q-Network and its convergence and final reward are evaluated. The states containing image and several scalars are processed by neural network that joints the two state parts into a unique flow. This neural network is named as Joint Neural Network (JNN) [1]. The training and test results show that the agent can successfully avoid any obstacles in the environment. In training, there exist some episodes crashed at the beginning of the training session because the random drones moves randomly in the environment and thus they can hit the learner drone during training. The test results are very promising and the learner drone reaches the destination with a success rate %100 in first two tests and with a success rate %98 in the last test.

Drone to Obstacle Distance Estimation Using YOLO V3 Network and Mathematical Principles

“…A Deep Reinforcement Learning (DRL) model is proposed by Cetin et.al (2019) [11] for the navigation of drones. With the help of Airsim simulator and Unreal Engine, several obstacles such as trees, parked cars, other moving drones, and houses are created for experimentation.…”

Counter a Drone and the Performance Analysis of Deep Reinforcement Learning Method and Human Pilot

“…In this paper, DDQN algorithm is chosen for DRL training. The best performance of DDQN algorithm have already been presented in previous studies [22], [6]. DRL model have been changed and the improvements are included considering the new image state which is improved by using the state of the art object detection algorithm and fences on the image state, new reward function and drone actions, updated geo-fence locations.…”

Visual Navigation System for Autonomous Drone using Fiducial Marker Detection

“…The drone can be navigated autonomously or by a human pilot. There are several works about developing the navigation algorithm for the drone [8], [9], [10], [11]. Hodge et al [12] presented a generic navigation algorithm that utilizes onboard sensors' data of the drone to navigate the drone to the target.…”