Drone Deep Reinforcement Learning: A Review

Azar, Ahmad Taher; Koubâa, Anis; Mohamed, Nada Ali; Ibrahim, Habiba; Ibrahim, Zahra Fathy; Kazim, Muhammad; Ammar, Adel; Benjdira, Bilel; Khamis, Alaa; Hameed, Ibrahim A.; Casalino, Gabriella

doi:10.3390/electronics10090999

Cited by 182 publications

(100 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement learning is a machine learning based method that can be applied to dynamic environments, and allows the algorithm to improve its performance by interacting with the environment [31]. Deep reinforcement learning-based methods have recently gathered great success in several domains [32][33][34][35], and they are extremely useful when there is a complex objective function such as in training autonomous vehicles [36] and drone control [37]. We believe that such an approach will make the system more robust to changes.…”

Section: Discussionmentioning

confidence: 99%

Vision-Less Sensing for Autonomous Micro-Drones

Pikalov

Azaria

Sonnenberg

et al. 2021

Sensors

View full text Add to dashboard Cite

This work presents a concept of intelligent vision-less micro-drones, which are motivated by flying animals such as insects, birds, and bats. The presented micro-drone (named BAT: Blind Autonomous Tiny-drone) can perform bio-inspired complex tasks without the use of cameras. The BAT uses LIDARs and self-emitted optical-flow in order to perform obstacle avoiding and maze-solving. The controlling algorithms were implemented on an onboard micro-controller, allowing the BAT to be fully autonomous. We further present a method for using the information collected by the drone to generate a detailed mapping of the environment. A complete model of the BAT was implemented and tested using several scenarios both in simulation and field experiments, in which it was able to explore and map complex building autonomously even in total darkness.

show abstract

Section: Discussionmentioning

confidence: 99%

Vision-Less Sensing for Autonomous Micro-Drones

Pikalov

Azaria

Sonnenberg

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The rapid development and pervasive use of UAVs together with the continuous growth of e-commerce significantly increased the relevance of this transportation mode in logistics and distribution systems [27,28]. Murray and Chu (2015) performed one of the first works investigating the use of drone-carrying vehicles in transportation systems [29].…”

Section: Delivery By Dronementioning

confidence: 99%

On Optimizing a Multi-Mode Last-Mile Parcel Delivery System with Vans, Truck and Drone

et al. 2021

View full text Add to dashboard Cite

This work focuses on the optimization of a last-mile delivery system with multiple transportation modes. In this scenario, parcels need to be delivered to each customer point. The major feature of the problem is the combination of a fleet of road vehicles (vans) with a drone. Each van visits a subset of demand nodes to be determined according to the route of the van. The drone serves the customers not served by vans. At the same time, considering the safety, policy and terrain as well as the need to replace the battery, the drone needs to be transported by truck to the identified station along with the parcel. From each such station, the drone serves a subset of customers according to a direct assignment pattern, i.e., every time the drone is launched, it serves one demand node and returns to the station to collect another parcel. Similarly, the truck is used to transport the drone and cargo between stations. This is somewhat different from the research of other scholars. In terms of the joint distribution of the drone and road vehicle, most scholars will choose the combination of two transportation tools, while we use three. The drone and vans are responsible for distribution services, and the trucks are responsible for transporting the goods and drone to the station. The goal is to optimize the total delivery cost which includes the transportation costs for the vans and the delivery cost for the drone. A fixed cost is also considered for each drone parking site corresponding to the cost of positioning the drone and using the drone station. A discrete optimization model is presented for the problem in addition to a two-phase heuristic algorithm. The results of a series of computational tests performed to assess the applicability of the model and the efficiency of the heuristic are reported. The results obtained show that nearly 10% of the cost can be saved by combining the traditional delivery mode with the use of a drone and drone stations.

show abstract

“…Additionally, deep learning has been used to detect multiple objects of interest, improve navigation [45] and obtain 6DOF positioning [46] or relative positioning [5,23]. Even reinforcement learning has been used to enhance flight precision [39,47]. However, for these processes to run in real-time, specialised cards such as GPUs or smart cameras are required to speed up the execution and distribution of the processes.…”

Section: Related Workmentioning

confidence: 99%

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

Rojas-Perez

Martínez-Carranza

2021

Sensors

View full text Add to dashboard Cite

Recent advances have shown for the first time that it is possible to beat a human with an autonomous drone in a drone race. However, this solution relies heavily on external sensors, specifically on the use of a motion capture system. Thus, a truly autonomous solution demands performing computationally intensive tasks such as gate detection, drone localisation, and state estimation. To this end, other solutions rely on specialised hardware such as graphics processing units (GPUs) whose onboard hardware versions are not as powerful as those available for desktop and server computers. An alternative is to combine specialised hardware with smart sensors capable of processing specific tasks on the chip, alleviating the need for the onboard processor to perform these computations. Motivated by this, we present the initial results of adapting a novel smart camera, known as the OpenCV AI Kit or OAK-D, as part of a solution for the ADR running entirely on board. This smart camera performs neural inference on the chip that does not use a GPU. It can also perform depth estimation with a stereo rig and run neural network models using images from a 4K colour camera as the input. Additionally, seeking to limit the payload to 200 g, we present a new 3D-printed design of the camera’s back case, reducing the original weight 40%, thus enabling the drone to carry it in tandem with a host onboard computer, the Intel Stick compute, where we run a controller based on gate detection. The latter is performed with a neural model running on an OAK-D at an operation frequency of 40 Hz, enabling the drone to fly at a speed of 2 m/s. We deem these initial results promising toward the development of a truly autonomous solution that will run intensive computational tasks fully on board.

show abstract

Drone Deep Reinforcement Learning: A Review

Cited by 182 publications

References 75 publications

Vision-Less Sensing for Autonomous Micro-Drones

Vision-Less Sensing for Autonomous Micro-Drones

On Optimizing a Multi-Mode Last-Mile Parcel Delivery System with Vans, Truck and Drone

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

Contact Info

Product

Resources

About