TransVLAD: Multi-Scale Attention-Based Global Descriptors for Visual Geo-Localization

Xu, Yifan; Shamsolmoali, Pourya; Granger, Éric; Nicodème, Claire; Gardès, Laurent; Yang, Jie

doi:10.1109/wacv56688.2023.00286

Cited by 13 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have shown that multiscale information can be utilized to effectively improve performance of a model in the VPR task 10,13,18,29,40,41 ; this approach can effectively avoid the problem of localization failure caused by the scale difference between the query image and the reference image. The differences between these approaches are how the multiscale information is obtained and how to collect more spatial contextual information when considering multiscale information.…”

Section: Leveraging Multiscale Informationmentioning

confidence: 99%

“…Peng et al 40 used a semantically enhanced local weighting scheme for local feature refinement and then constructed an attention pyramid based on the spatial saliency of regional features for the adaptive encoding of local features. Xu et al 41 used an attention-based sparse encoder to obtain feature maps, thereby capturing global dependencies. Then, self-supervised learning was utilized to further acquire multiscale information between query images, effectively reducing the visual ambiguities arising in large-scale VPR.…”

Section: Leveraging Multiscale Informationmentioning

confidence: 99%

See 1 more Smart Citation

Convolutional MLP orthogonal fusion of multiscale features for visual place recognition

Gan,

Zhou,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Visual place recognition (VPR) involves obtaining robust image descriptors to cope with differences in camera viewpoints and drastic external environment changes. Utilizing multiscale features improves the robustness of image descriptors; however, existing methods neither exploit the multiscale features generated during feature extraction nor consider the feature redundancy problem when fusing multiscale information when image descriptors are enhanced. We propose a novel encoding strategy—convolutional multilayer perceptron orthogonal fusion of multiscale features (ConvMLP-OFMS)—for VPR. A ConvMLP is used to obtain robust and generalized global image descriptors and the multiscale features generated during feature extraction are used to enhance the global descriptors to cope with changes in the environment and viewpoints. Additionally, an attention mechanism is used to eliminate noise and redundant information. Compared to traditional methods that use tensor splicing for feature fusion, we introduced matrix orthogonal decomposition to eliminate redundant information. Experiments demonstrated that the proposed architecture outperformed NetVLAD, CosPlace, ConvAP, and other methods. On the Pittsburgh and MSLS datasets, which contained significant viewpoint and illumination variations, our method achieved 92.5% and 86.5% Recall@1, respectively. We also achieved good performances—80.6% and 43.2%—on the SPED and NordLand datasets, respectively, which have more extreme illumination and appearance variations.

show abstract

Section: Leveraging Multiscale Informationmentioning

confidence: 99%

Section: Leveraging Multiscale Informationmentioning

confidence: 99%

Convolutional MLP orthogonal fusion of multiscale features for visual place recognition

Gan,

Zhou,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…The ViT [19] model utilizes the classic Transformer encoder structure to achieve image classification tasks, marking the beginning of the Transformer's application in the field of vision and gradually playing a role in cross-view geo-localization [57]. Following the architecture of NetVLAD [58], TransVLAD [15] utilizes a sparse transformer encoder to obtain global descriptors. It was further combined with DFM [59] to obtain more dense and accurate matching results.…”

Section: B Transformer In Geo-localizationmentioning

confidence: 99%

“…The remarkable contextual modeling capability of the transformer compensates for the limitations of CNNs. At present, transformer-based cross-view geo-localization technology mainly utilizes transformer encoders as the backbone of feature extraction, improving the ability of contextual feature extraction [15], [16], [17], [18]. Some methods use ViT [19] as the backbone for extracting context-sensitive information [20], [21] to better adapt to image data.…”

Section: Introductionmentioning

confidence: 99%

GeoFormer: An Effective Transformer-Based Siamese Network for UAV Geolocalization

Li,

Yang,

Fan

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Cross-view geo-localization of unmanned aerial vehicles (UAVs) is a challenging task due to the positional discrepancies and uncertainties in scale and distance between UAVs and satellite views. Existing transformer-based geolocalization methods mainly use encoders to mine image contextual information. However, these methods have some limitations when dealing with scale changes in cross-view images. Therefore, we present an effective transformer-based Siamese network tailored for UAV geo-localization, called GeoFormer. Firstly, an efficient transformer feature extraction network was designed, which utilizes linear attention to reduce the computational complexity and improve the computational efficiency of the network. Among them, we designed an efficient separable perceptron module based on depth-wise separable convolution, which can effectively reduce the computational cost while improving the feature representation of the network. Secondly, we proposed a multi-scale feature aggregation module (MFAM), which deeply fuses salient features at different scales through a feed-forward neural network to generate global feature representations with rich semantics, which improves the model's ability to capture image details and represent robust features. Additionally, we designed a semantic-guided region segmentation module (SRSM), which utilizes a k-modes clustering algorithm to divide the feature map into multiple regions with semantic consistency and performs feature recognition within each semantic region to improve the accuracy of image matching. Finally, we designed a hierarchical reinforcement rotation matching strategy to achieve accurate UAV geo-localization based on the retrieval results of UAV view query satellite images using SuperPoint keypoints extraction and LightGlue rotation matching. According to the experimental results, our method effectively achieves UAV geo-localization.

show abstract

“…NetVLAD is a differentiable implementation of VLAD that is trained end-to-end with a CNN backbone for direct place recognition. It has been widely adopted in various works [9], [11], [14], [24]. Task-specific patch-level features have also been explored for VPR [9], [25].…”

Section: A Recent Progress On Visual Place Recognition (Vpr)mentioning

confidence: 99%

Weak-supervised Visual Geo-localization via Attention-based Knowledge Distillation

Shamsolmoali

Yang

2022

2022 26th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Visual place recognition (VPR) is a highly challenging task that has a wide range of applications, including robot navigation and self-driving vehicles. VPR is particularly difficult due to the presence of duplicate regions and the lack of attention to small objects in complex scenes, resulting in recognition deviations. In this paper, we present ClusVPR, a novel approach that tackles the specific issues of redundant information in duplicate regions and representations of small objects. Different from existing methods that rely on Convolutional Neural Networks (CNNs) for feature map generation, ClusVPR introduces a unique paradigm called Clustering-based Weighted Transformer Network (CWTNet). CWTNet leverages the power of clustering-based weighted feature maps and integrates global dependencies to effectively address visual deviations encountered in large-scale VPR problems. We also introduce the optimized-VLAD (OptLAD) layer that significantly reduces the number of parameters and enhances model efficiency. This layer is specifically designed to aggregate the information obtained from scale-wise image patches. Additionally, our pyramid selfsupervised strategy focuses on extracting representative and diverse information from scale-wise image patches instead of entire images, which is crucial for capturing representative and diverse information in VPR. Extensive experiments on four VPR datasets show our model's superior performance compared to existing models while being less complex.

show abstract

TransVLAD: Multi-Scale Attention-Based Global Descriptors for Visual Geo-Localization

Cited by 13 publications

References 31 publications

Convolutional MLP orthogonal fusion of multiscale features for visual place recognition

Convolutional MLP orthogonal fusion of multiscale features for visual place recognition

GeoFormer: An Effective Transformer-Based Siamese Network for UAV Geolocalization

Weak-supervised Visual Geo-localization via Attention-based Knowledge Distillation

Contact Info

Product

Resources

About