Finding Point with Image: An End-to-End Benchmark for Vision-based UAV Localization

Abstract

In the past, image retrieval was the mainstream solution for cross-view geolocation and UAV visual localization tasks. In a nutshell, the way of image retrieval is to obtain the final required information, such as GPS, through a transitional perspective. However, the way of image retrieval is not completely end-to-end. And there are some redundant operations such as the need to prepare the feature library in advance, and the sampling interval problem of the gallery construction, which make it difficult to implement large-scale applications. In this article we propose an end-to-end positioning scheme, Finding Point with Image (FPI), which aims to directly find the corresponding location in the image of source B (satellite-view) through the image of source A (drone-view). To verify the feasibility of our framework, we construct a new dataset (UL14), which is designed to solve the UAV visual self-localization task. At the same time, we also build a transformer-based baseline to achieve end-to-end training. In addition, the previous evaluation methods are no longer applicable under the framework of FPI. Thus, Metrelevel Accuracy (MA) and Relative Distance Score (RDS) are proposed to evaluate the accuracy of UAV localization. At the same time, we preliminarily compare FPI and image retrieval method, and the structure of FPI achieves better performance in both speed and efficiency. In particular, the task of FPI remains great challenges due to the large differences between different views and the drastic spatial scale transformation.

OS-FPI: A Coarse-to-Fine One-Stream Network for UAV Geolocalization

“…Given a picture of a UAV in any area, the current location of the UAV can be found in the database. To this end, [15] proposed a new end-to-end method of finding points with images and a new UL14 dataset, where the authors used a two-stream network without shared weights to extract the vertical view of the UAV image and the satellite image respectively, after which the response map was obtained by relational modeling, and the point with the largest response value in the map was the current position of the UAV image predicted by the model. This endto-end approach provides a completely new direction for the development of UAV visual localization, and the authors also propose an MA metric to quantify error, using meters as the unit of measurement for error.…”

“…This endto-end approach provides a completely new direction for the development of UAV visual localization, and the authors also propose an MA metric to quantify error, using meters as the unit of measurement for error. On the basis of FPI [15], in order to alleviate the multi-scale problem in the task of finding points with images, the author proposed the WAMF module [16]. And the final output response map is restored to the original satellite map size, thus reducing the problem of inaccurate positioning due to the small resolution of the prediction map and further improving the positioning accuracy of the model.…”

“…Before the introduction of offset prediction, previous methods, such as FPI [15] and WAMF-FPI [16], relied on the point with the largest value in the heat map to determine the location of the center of the UAV image. As shown in the heat map in Figure 4(a), the point with the largest value on the map is the current UAV position predicted by the model.…”

“…The ongoing advancement in computer vision technologies, such as object detection, image retrieval, and object tracking, offer the potential of relying solely on visual information for UAV geolocation and navigation tasks. Current cross-view geolocation technology for UAVs is mainly realized through two approaches: image retrieval [11]- [14] and the method of finding points with images [15], [16].…”

“…In order to solve the problems of poor positioning accuracy and time-consuming application, a new method of finding points with images was proposed [15], [16]. It borrows from the field of object tracking by modeling the relationship between UAV images in vertical view and satellite images to obtain a response map.…”