Semantic Match Consistency for Long-Term Visual Localization

Toft, Carl; Stenborg, Erik; Hammarstrand, Lars; Brynte, Lucas; Pollefeys, Marc; Sattler, Torsten; Kahl, Fredrik

doi:10.1007/978-3-030-01216-8_24

Cited by 124 publications

(133 citation statements)

References 56 publications

Supporting

Mentioning

131

Contrasting

Order By: Relevance

“…The latter type of methods have recently been shown to not perform consistently better than image retrieval methods [76], i.e., approaches that approximate the pose of the query image by the pose of the most similar database image [3,38,87]. As such, state-of-the-art methods for long-term visual localization at scale either rely on local features for matching [28,71,78,83,85,86] or use image retrieval techniques [2-4, 63, 80, 87, 94].…”

Section: Related Workmentioning

confidence: 99%

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

Larsson

Stenborg

Toft

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Long-term visual localization is the problem of estimating the camera pose of a given query image in a scene whose appearance changes over time. It is an important problem in practice, for example, encountered in autonomous driving. In order to gain robustness to such changes, long-term localization approaches often use segmantic segmentations as an invariant scene representation, as the semantic meaning of each scene part should not be affected by seasonal and other changes. However, these representations are typically not very discriminative due to the limited number of available classes. In this paper, we propose a new neural network, the Fine-Grained Segmentation Network (FGSN), that can be used to provide image segmentations with a larger number of labels and can be trained in a self-supervised fashion. In addition, we show how FGSNs can be trained to output consistent labels across seasonal changes. We demonstrate through extensive experiments that integrating the fine-grained segmentations produced by our FGSNs into existing localization algorithms leads to substantial improvements in localization performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

Larsson

Stenborg

Toft

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…IBL has witnessed tremendous advancement by means of deep learning [18,19] and image retrieval techniques [1,2,34]. However, structure-based IBL [6,21,23,31,37,38,41] by directly establishing 2D-3D matches between a query image and SfM models is still the most prevailing strategy. Recent state-of-the-art methods handle the match ambiguity under high-dimensional feature representation with semantic consistency [38].…”

Section: Introductionmentioning

confidence: 99%

“…In feature-wise filtering, we reformulate a traditional match scoring function [16] with a bilateral Hamming ratio test to better evaluate the distinctiveness of matches. In visibility-arXiv:1908.06141v1 [cs.CV] 16 Aug 2019 Method Feature Type Compactness Match Filtering Prior-free SR Feature-wise Visibility-wise Geometry-wise AS [31] SIFT Strict WPE [21] SIFT Relaxed CSL [37] SIFT Relaxed * CPV [41] SIFT Relaxed * Hyperpoints [29] SIFT Relaxed In RPE EGM [23] SIFT+Binary Relaxed TC [6] SIFT Relaxed SMC [38] SIFT Relaxed * Our method Binary Relaxed Before RPE Table 1: Comparison between our method and other structure-based IBL methods. * means that the vertical direction of camera is known in advance, SR represents Spatial Reconfiguration and RPE represents RANSAC-based Pose Estimation.…”

Section: Introductionmentioning

confidence: 99%

“…The matches that are co-visible frequently with seed matches are prioritized to accelerate the matching process. Recent works [6,21,28,37,38,41] commonly relax the feature-wise filtering criterion to preserve more correct matches and shift the filtering task to visibility or geometry tools. Li et al introduce a RANSAC sampling strategy by prioritizing samples with frequent co-visibility [21].…”

Section: Introductionmentioning

confidence: 99%

“…Assuming that the gravity direction and an approximate estimation of camera height are known, both Zeisl et al and Svarm et al present geometric outlier filtering approaches to handle extremely large outlier ratios [6,37]. Toft et al derive an outlier filtering method by combining the known gravity direction prior and semantic information [38].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

Cheng

Wang

Chen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Image-based localization (IBL) aims to estimate the 6DOF camera pose for a given query image. The camera pose can be computed from 2D-3D matches between a query image and Structure-from-Motion (SfM) models. Despite recent advances in IBL, it remains difficult to simultaneously resolve the memory consumption and match ambiguity problems of large SfM models. In this work, we propose a cascaded parallel filtering method that leverages the feature, visibility and geometry information to filter wrong matches under binary feature representation. The core idea is that we divide the challenging filtering task into two parallel tasks before deriving an auxiliary camera pose for final filtering. One task focuses on preserving potentially correct matches, while another focuses on obtaining high quality matches to facilitate subsequent more powerful filtering. Moreover, our proposed method improves the localization accuracy by introducing a quality-aware spatial reconfiguration method and a principal focal length enhanced pose estimation method. Experimental results on real-world datasets demonstrate that our method achieves very competitive localization performances in a memoryefficient manner.

show abstract

VISEL: A visual and magnetic fusion‐based large‐scale indoor localization system with improved high‐precision semantic maps

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

show abstract

Semantic Match Consistency for Long-Term Visual Localization

Cited by 124 publications

References 56 publications

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

VISEL: A visual and magnetic fusion‐based large‐scale indoor localization system with improved high‐precision semantic maps

Contact Info

Product

Resources

About