Cross-View Geo-Localization via Learning Disentangled Geometric Layout Correspondence

Abstract

Cross-view geo-localization aims to estimate the location of a query ground image by matching it to a reference geo-tagged aerial images database. As an extremely challenging task, its difficulties root in the drastic view changes and different capturing time between two views. Despite these difficulties, recent works achieve outstanding progress on cross-view geo-localization benchmarks. However, existing methods still suffer from poor performance on the cross-area benchmarks, in which the training and testing data are captured from two different regions. We attribute this deficiency to the lack of ability to extract the spatial configuration of visual feature layouts and models' overfitting on low-level details from the training set. In this paper, we propose GeoDTR which explicitly disentangles geometric information from raw features and learns the spatial correlations among visual features from aerial and ground pairs with a novel geometric layout extractor module. This module generates a set of geometric layout descriptors, modulating the raw features and producing high-quality latent representations. In addition, we elaborate on two categories of data augmentations, (i) Layout simulation, which varies the spatial configuration while keeping the low-level details intact. (ii) Semantic augmentation, which alters the low-level details and encourages the model to capture spatial configurations. These augmentations help to improve the performance of the cross-view geo-localization models, especially on the cross-area benchmarks. Moreover, we propose a counterfactual-based learning process to benefit the geometric layout extractor in exploring spatial information. Extensive experiments show that GeoDTR not only achieves state-of-the-art results but also significantly boosts the performance on same-area and cross-area benchmarks. Our code can be found at https://gitlab.com/vail-uvm/geodtr.

Fine-Grained Permeable Surface Mapping through Parallel U-Net

“…Rashkovetsky et al combined multiple U-Nets [11] to detect and segment wildfires from multi-band satellite images [6]. Zhang et al proposed to identify the location of ground images by comparing the learned features from geo-tagged satellite images [25,26]. Li et al studied poverty mapping from satellite images by proposing a point-to-region dynamic learning framework that can help people find clean water and sanitation services in low-income countries [27].…”

“…Different from some of the methods mentioned above, for example, refs. [25,27], which only use color channel information, considering the nature of permeable surface mapping, we also take advantage of the infrared band. Our extensive experiments and ablation studies in Section 4 demonstrate the effectiveness of the infrared band in our proposed model.…”

GeoFormer: An Effective Transformer-Based Siamese Network for UAV Geolocalization

“…L2LTR [16] employs a transformer encoder as a backbone, utilizing self and cross attention mechanisms to emulate global dependency relationships between adjacent layers, thereby enhancing the quality of the learned representations. GeoDTR [17] utilizes a transformer encoder to separate geometric information from the original features and, through a novel geometric contextual extraction module, it can learn the spatial correlations between visual features in satellite and ground images. TransGeo [18] fully leverages the advantages of transformer encoder global information modeling and explicit positional encoding, reducing computational costs and enhancing performance.…”

“…The remarkable contextual modeling capability of the transformer compensates for the limitations of CNNs. At present, transformer-based cross-view geo-localization technology mainly utilizes transformer encoders as the backbone of feature extraction, improving the ability of contextual feature extraction [15], [16], [17], [18]. Some methods use ViT [19] as the backbone for extracting context-sensitive information [20], [21] to better adapt to image data.…”