Visual localization with a monocular camera for unmanned aerial vehicle based on landmark detection and tracking using YOLOv5 and DeepSORT

Abstract

Absolute visual localization is of significant importance for unmanned aerial vehicles when the satellite-based localization system is not available. With the rapid evolution in the field of deep learning, the real-time visual detection and tracking of landmarks by an unmanned aerial vehicle could be implemented onboard. This study demonstrates a landmark-based visual localization framework for unmanned aerial vehicles flying at low altitudes. YOLOv5 and DeepSORT are used for multi-object detection and tracking, respectively. The unmanned aerial vehicle localization is achieved according to the geometric similarity between the geotagged transmission towers and the annotated images captured by a monocular camera. The validation is accomplished both in the Rflysim-based simulation and the quadrotor-based real flight. The localization precision is about 10 m, and the location update frequency reaches 5 Hz with a commercially available entry-level edge artificial intelligence platform. The proposed visual localization strategy needs no satellite image as a reference map, which saves a significant amount of the GPU memory and makes possible the end-to-end implementation on small unmanned aerial vehicles.

Joint frame rate adaptation and object recognition model selection for stabilized unmanned aerial vehicle surveillance

“…These characteristics of YOLOv5 make it suitable for our application because UAVs have limited power and computational resources. A real‐world flight experiment was conducted using a quadcopter UAV flying at low altitude and using YOLOv5 for object recognition [32].…”

From Pixels to Precision: A Survey of Monocular Visual Odometry in Digital Twin Applications

“…Visual odometry (VO) and VSLAM are two closely related techniques that are used to determine a robot or machine’s location and orientation through the analysis of corresponding camera images. Both techniques can utilize a monocular camera, but they have distinct characteristics and objectives [ 6 , 7 , 8 ].…”

Monocular Vision Guidance for Unmanned Surface Vehicle Recovery

“…Multi-vision requires multiple cameras for shooting, resulting in higher costs, and also requires addressing the feature matching issue from different cameras, which leads to complex operations [16,17]. In contrast, monocular vision only requires a single camera, enabling implementation through the pinhole imaging principle, resulting in lower costs and convenient operation [18,19]. The commonly used methods for monocular vision positioning include the Perspective-n-Point (PNP) method [20], the 2 of 14 imaging model method [21], the data regression modeling method [22], and the geometric relationship method [23].…”