Deep learning for vision-based micro aerial vehicle autonomous landing

Abstract

Vision-based techniques are widely used in micro aerial vehicle autonomous landing systems. Existing vision-based autonomous landing schemes tend to detect specific landing landmarks by identifying their straightforward visual features such as shapes and colors. Though efficient to compute, these schemes only apply to landmarks with limited variability and require strict environmental conditions such as consistent lighting. To overcome these limitations, we propose an end-to-end landmark detection system based on a deep convolutional neural network, which not only easily scales up to a larger number of various landmarks but also exhibit robustness to different lighting conditions. Furthermore, we propose a separative implementation strategy which conducts convolutional neural network training and detection on different hardware platforms separately, i.e. a graphics processing unit work station and a micro aerial vehicle on-board system, subject to their specific implementation requirements. To evaluate the performance of our framework, we test it on synthesized scenarios and real-world videos captured by a quadrotor on-board camera. Experimental results validate that the proposed vision-based autonomous landing system is robust to landmark variability in different backgrounds and lighting situations.

SlimDeblurGAN-Based Motion Deblurring and Marker Detection for Autonomous Drone Landing

“…In addition, they introduced Profile Checker V2 to improve accuracy. As a result, their method could operate with a maximum range of 50 m. Similarly, Yu et al [ 14 ] introduced a deep-learning-based method for MAV autonomous landing systems, and they adopted a variant of the YOLO detector to detect landmarks. The system achieved high accuracy of marker detection and exhibited robustness to various conditions, such as variations in landmarks under different lighting conditions and backgrounds.…”

“…However, its target was only for logo detection, which was different from our research of marker detection by a drone camera. Although the method in [ 13 , 14 , 21 ] achieved a 99% accuracy for landmark or marker, based on field experiments, they assumed only the slow movement or landing of drone, which did not generate the motion blurring. However, in the actual case of drone movement or landing at normal speed, motion blurring occurred frequently, as mentioned in [ 20 ].…”

“…In object detection, traditional handcrafted feature-based methods showed limited performance [1][2][3][4][5][6][7][8][9][10][11][12]. A competitive approach is to apply deep-learning-based methods, which have gained popularity in recent years [13][14][15][16]. However, deploying deep learning models to a UAV onboard system raises new challenges-(1) difficulties of scene parsing in cases of low-resolution or motion-blurred input, (2) difficulties of deploying the model to an embedded system with limited memory and computation power, and (3) balancing between model accuracy and execution time.…”

Deep Learning-Based Super-Resolution Reconstruction and Marker Detection for Drone Landing

“…Their method can detect the accurate center and direction of a marker using a Profile Checker v2, and the drone can be operated for very large distances, with 50 m being the furthest. Similarly, Yu et al proposed a visionguided autonomous landing system of MAVs based on a modified SqueezeNet and a you only look once (YOLO) model for detecting landmarks [17]. The system is robust to variations in landmarks under different lighting conditions and backgrounds.…”

Editorial for special collection on the estimation and control of MAV navigation in GPS-denied cluttered environments

“…Yu et al. 3 propose an end-to-end landmark detection system based on a deep convolutional neural network and an associated embedded implementation on a graphics implementation processing unit to perform vision-based autonomous landing.…”