Cross-View Image Matching for Geo-Localization in Urban Environments

Abstract

In this paper, we address the problem of cross-view image geo-localization. Specifically, we aim to estimate the GPS location of a query street view image by finding the matching images in a reference database of geotagged bird's eye view images, or vice versa. To this end, we present a new framework for cross-view image geolocalization by taking advantage of the tremendous success of deep convolutional neural networks (CNNs) in image classification and object detection. First, we employ the Faster R-CNN [16] to detect buildings in the query and reference images. Next, for each building in the query image, we retrieve the k nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs. To find the correct NN for each query building, we develop an efficient multiple nearest neighbors matching method based on dominant sets. We evaluate the proposed framework on a new dataset that consists of pairs of street view and bird's eye view images. Experimental results show that the proposed method achieves better geo-localization accuracy than other approaches and is able to generalize to images at unseen locations.

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

“…The database serves as the index of an image retrieval system [28,29,30,31,32,33] or the training data of a landmark classifier [34,35,36]. Cross-view geolocation recognition makes additional use of satellite or aerial imagery to determine query locations [37,38,39,40].…”

Bridging the Domain Gap for Ground-to-Aerial Image Matching

“…Bansal et al [4] explored facade matching. Tian et al [37] matched building features in oblique views. Recent work by [18] exploit the NetVLAD [3] to obtain view-invariant descriptors for cross-view pairs and use them for matching.…”

Co‐registration of panoramic mobile mapping images and oblique aerial images

“…Image registration is a research topic that is widely covered in the literature. Although the field is moving towards machine learning and is able to cope with wide‐baseline problems (Shan et al., ; Lin et al., ; Zagoruyko and Komodakis, ; Melekhov et al., ; Tian et al., ), these approaches are still very experimental and cannot provide the reliability and registration accuracy required for the task at hand. Alternatively, authors rely on synthesised views to homogenise the datasets to be registered (Morel and Yu, ; Roth et al., ).…”