Perception, Guidance, and Navigation for Indoor Autonomous Drone Racing Using Deep Learning

Jung, Sunggoo; Hwang, Sangyeon; Shin, Heemin; Shim, David Hyunchul

doi:10.1109/lra.2018.2808368

Cited by 152 publications

(96 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Jung et al [8] consider the problem of autonomous drone navigation in a previously unseen track. They use line-ofsight guidance combined with a deep-learning-based gate detector.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing

Kaufmann

Gehrig

Foehn

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

107

View full text Add to dashboard Cite

Autonomous micro aerial vehicles still struggle with fast and agile maneuvers, dynamic environments, imperfect sensing, and state estimation drift. Autonomous drone racing brings these challenges to the fore. Human pilots can fly a previously unseen track after a handful of practice runs. In contrast, state-of-the-art autonomous navigation algorithms require either a precise metric map of the environment or a large amount of training data collected in the track of interest. To bridge this gap, we propose an approach that can fly a new track in a previously unseen environment without a precise map or expensive data collection. Our approach represents the global track layout with coarse gate locations, which can be easily estimated from a single demonstration flight. At test time, a convolutional network predicts the poses of the closest gates along with their uncertainty. These predictions are incorporated by an extended Kalman filter to maintain optimal maximum-aposteriori estimates of gate locations. This allows the framework to cope with misleading high-variance estimates that could stem from poor observability or lack of visible gates. Given the estimated gate poses, we use model predictive control to quickly and accurately navigate through the track. We conduct extensive experiments in the physical world, demonstrating agile and robust flight through complex and diverse previouslyunseen race tracks. The presented approach was used to win the IROS 2018 Autonomous Drone Race Competition, outracing the second-placing team by a factor of two.

show abstract

“…Jung et al [8] consider the problem of autonomous drone navigation in a previously unseen track. They use line-ofsight guidance combined with a deep-learning-based gate detector.…”

Section: Related Workmentioning

confidence: 99%

“…Our approach addresses the limitations of both works [9], [8]. It operates reliably even when no gate is in sight, while eliminating the need to retrain the perception system for every new track.…”

Section: Related Workmentioning

confidence: 99%

Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing

Kaufmann

Gehrig

Foehn

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

107

View full text Add to dashboard Cite

show abstract

“…Some methods learn to navigate from raw images, indicating the next movement of the UAV, thus being able to navigate avoiding obstacles [49,59,62,69]. Other authors proposed specific solutions for certain applications that involve following a trail or a gate [55,70]. However, such methods do not include a collision avoidance module.…”

Section: On the Application Of DL To Uavsmentioning

confidence: 99%

“…Other outdoor UAV solutions are trained for urban spaces, such as CNN-based models that detect all the potential obstacles [72] or that control a UAV through the streets of a city environment [62]. [70] CNN The network detects the gate center. Then, Indoors an external guidance algorithm is applied.…”

Section: On the Application Of DL To Uavsmentioning

confidence: 99%

A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance

et al. 2019

View full text Add to dashboard Cite

Advances in Unmanned Aerial Vehicles (UAVs), also known as drones, offer unprecedented opportunities to boost a wide array of large-scale Internet of Things (IoT) applications. Nevertheless, UAV platforms still face important limitations mainly related to autonomy and weight that impact their remote sensing capabilities when capturing and processing the data required for developing autonomous and robust real-time obstacle detection and avoidance systems. In this regard, Deep Learning (DL) techniques have arisen as a promising alternative for improving real-time obstacle detection and collision avoidance for highly autonomous UAVs. This article reviews the most recent developments on DL Unmanned Aerial Systems (UASs) and provides a detailed explanation on the main DL techniques. Moreover, the latest DL-UAV communication architectures are studied and their most common hardware is analyzed. Furthermore, this article enumerates the most relevant open challenges for current DL-UAV solutions, thus allowing future researchers to define a roadmap for devising the new generation affordable autonomous DL-UAV IoT solutions.

show abstract

“…Application-specific customisation has also been proposed for single-shot detectors, utilising prior domain knowledge to adapt the model in a controlled manner that improves performance, with minimum effect on accuracy. For example, in [18] a single-shot detector is employed for gate detection in drone racing. Considering the simple geometry of square gates used in drone racing competitions, plenty of the unnecessary high-level feature layers are removed.…”

Section: B Efficient Learning-based Detectors On Uavsmentioning

confidence: 99%

Informed Region Selection for Efficient UAV-based Object Detectors: Altitude-aware Vehicle Detection with CyCAR Dataset

Kouris

Kyrkou

Bouganis

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Deep Learning-based object detectors enhance the capabilities of remote sensing platforms, such as Unmanned Aerial Vehicles (UAVs), in a wide spectrum of machine vision applications. However, the integration of deep learning introduces heavy computational requirements, preventing the deployment of such algorithms in scenarios that impose lowlatency constraints during inference, in order to make missioncritical decisions in real-time. In this paper, we address the challenge of efficient deployment of region-based object detectors in aerial imagery, by introducing an informed methodology for extracting candidate detection regions (proposals). Our approach considers information from the UAV on-board sensors, such as flying altitude and light-weight computer vision filters, along with prior domain knowledge to intelligently decrease the number of region proposals by eliminating false-positives at an early stage of the computation, reducing significantly the computational workload while sustaining the detection accuracy. We apply and evaluate the proposed approach on the task of vehicle detection. Our experiments demonstrate that state-of-the-art detection models can achieve up to 2.6x faster inference by employing our altitude-aware data-driven methodology. Alongside, we introduce and provide to the community a novel vehicle-annotated and altitude-stamped dataset of real UAV imagery, captured at numerous flying heights under a wide span of traffic scenarios.

show abstract

Perception, Guidance, and Navigation for Indoor Autonomous Drone Racing Using Deep Learning

Cited by 152 publications

References 15 publications

Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing

Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing

A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance

Informed Region Selection for Efficient UAV-based Object Detectors: Altitude-aware Vehicle Detection with CyCAR Dataset

Contact Info

Product

Resources

About