Efficient Optical Flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone

McGuire, Kimberly; Croon, Guido de; Wagter, Christophe De; Tuyls, Karl; Kappen, Hilbert J.

doi:10.1109/lra.2017.2658940

Cited by 169 publications

(105 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This could resolve the more complex collision scenarios, especially in smaller rooms. There is also on-going work to improve the miniaturized platforms to feature a front-facing camera, as in (McGuire et al 2017), and active wall-sensors, so as to achieve fully autonomous teams of MAVs capable of exploring an unknown environment safely.…”

Section: Discussionmentioning

confidence: 99%

On-board communication-based relative localization for collision avoidance in Micro Air Vehicle teams

et al. 2018

View full text Add to dashboard Cite

To avoid collisions, Micro Air Vehicles (MAVs) flying in teams require estimates of their relative locations, preferably with minimal mass and processing burden. We present a relative localization method where MAVs need only to communicate with each other using their wireless transceiver. The MAVs exchange on-board states (velocity, height, orientation) while the signal strength indicates range. Fusing these quantities provides a relative location estimate. We used this for collision avoidance in tight areas, testing with up to three AR.Drones in a 4 m × 4 m area and with two miniature drones (≈ 50 g) in a 2 m × 2 m area. The MAVs could localize each other and fly several minutes without collisions. In our implementation, MAVs communicated using Bluetooth antennas. The results were robust to the high noise and disturbances in signal strength. They could improve further by using transceivers with more accurate signal strength readings.

show abstract

Section: Discussionmentioning

confidence: 99%

On-board communication-based relative localization for collision avoidance in Micro Air Vehicle teams

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Few previous works presented nano-size flying robots with some degree of autonomous navigation relying on onboard computation. In [13], the authors developed a 4 g stereocamera and proposed a velocity estimation algorithm able to run on the MCU on board a 40 g flying robot. If on one side this solution allows the drone to avoid obstacles during the flight, it still requires favorable flight conditions (e.g., low flight speed of 0.3 m/s).…”

Section: Related Workmentioning

confidence: 99%

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

Palossi

Loquercio

Conti

et al. 2019

IEEE Internet Things J.

151

117

View full text Add to dashboard Cite

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm 2 . In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resourceconstrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed aboard within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

show abstract

“…The traditional approach to the navigation of nano-drones requires to offload the computation to a remote basestation [1], [5], [7], demanding high-frequency video stream-ing, which lowers reliability and imposes constraints on maximum distance, introduces control latency and is poorly scalable. On the other hand, COTS nano-size quadrotors, like the Bitcraze Crazyflie 2.0 or the Walkera QR LadyBug, usually make use of very simple computing devices such as singlecore microcontroller units (MCUs) like the ST Microelectronics STM32F4 [1], [2], [8]. Autonomous flying capabilities achievable on these platforms are, to the date, very limited.…”

Section: Related Workmentioning

confidence: 99%

“…Autonomous flying capabilities achievable on these platforms are, to the date, very limited. In [2] the proposed obstacle avoidance functionality requires favorable flight conditions (e.g., low flight speed of 0.3 m/s). The solutions proposed in [3], [4] are limited to hovering and do not reach the accuracy of computationally expensive techniques leveraged by powerful standard-size UAVs.…”

Section: Related Workmentioning

confidence: 99%

An Open Source and Open Hardware Deep Learning-Powered Visual Navigation Engine for Autonomous Nano-UAVs

Palossi

Conti

Benini

2019

2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS)

View full text Add to dashboard Cite

Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27 g nano-UAV system able to run aboard an end-to-end, closedloop visual pipeline for autonomous navigation based on a stateof-the-art deep-learning algorithm, built upon the open-source CrazyFlie 2.0 nano-quadrotor. Our visual navigation engine is enabled by the combination of an ultra-low power computing device (the GAP8 system-on-chip) with a novel methodology for the deployment of deep convolutional neural networks (CNNs). We enable onboard real-time execution of a state-of-the-art deep CNN at up to 18 Hz. Field experiments demonstrate that the system's high responsiveness prevents collisions with unexpected dynamic obstacles up to a flight speed of 1.5 m/s. In addition, we also demonstrate the capability of our visual navigation engine of fully autonomous indoor navigation on a 113 m previously unseen path. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

show abstract

Efficient Optical Flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone

Cited by 169 publications

References 23 publications

On-board communication-based relative localization for collision avoidance in Micro Air Vehicle teams

On-board communication-based relative localization for collision avoidance in Micro Air Vehicle teams

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

An Open Source and Open Hardware Deep Learning-Powered Visual Navigation Engine for Autonomous Nano-UAVs

Contact Info

Product

Resources

About