City-Scale Localization for Cameras with Known Vertical Direction

Abstract

We consider the problem of localizing a novel image in a large 3D model, given that the gravitational vector is known. In principle, this is just an instance of camera pose estimation, but the scale of the problem introduces some interesting challenges. Most importantly, it makes the correspondence problem very difficult so there will often be a significant number of outliers to handle. To tackle this problem, we use recent theoretical as well as technical advances. Many modern cameras and phones have gravitational sensors that allow us to reduce the search space. Further, there are new techniques to efficiently and reliably deal with extreme rates of outliers. We extend these methods to camera pose estimation by using accurate approximations and fast polynomial solvers. Experimental results are given demonstrating that it is possible to reliably estimate the camera pose despite cases with more than 99 percent outlier correspondences in city-scale models with several millions of 3D points.

Two-View Orthographic Epipolar Geometry: Minimal and Optimal Solvers

“…The authors used this to produce algorithms for optimal image stitching and 2D-registration. In [24,25], the authors used similar methods to perform large-scale imagebased localization. We will here describe how these ideas can be applied to orthographic essential matrix estimation, resulting in an optimal method.…”

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

“…IBL has witnessed tremendous advancement by means of deep learning [18,19] and image retrieval techniques [1,2,34]. However, structure-based IBL [6,21,23,31,37,38,41] by directly establishing 2D-3D matches between a query image and SfM models is still the most prevailing strategy. Recent state-of-the-art methods handle the match ambiguity under high-dimensional feature representation with semantic consistency [38].…”

“…In feature-wise filtering, we reformulate a traditional match scoring function [16] with a bilateral Hamming ratio test to better evaluate the distinctiveness of matches. In visibility-arXiv:1908.06141v1 [cs.CV] 16 Aug 2019 Method Feature Type Compactness Match Filtering Prior-free SR Feature-wise Visibility-wise Geometry-wise AS [31] SIFT Strict WPE [21] SIFT Relaxed CSL [37] SIFT Relaxed * CPV [41] SIFT Relaxed * Hyperpoints [29] SIFT Relaxed In RPE EGM [23] SIFT+Binary Relaxed TC [6] SIFT Relaxed SMC [38] SIFT Relaxed * Our method Binary Relaxed Before RPE Table 1: Comparison between our method and other structure-based IBL methods. * means that the vertical direction of camera is known in advance, SR represents Spatial Reconfiguration and RPE represents RANSAC-based Pose Estimation.…”

“…The matches that are co-visible frequently with seed matches are prioritized to accelerate the matching process. Recent works [6,21,28,37,38,41] commonly relax the feature-wise filtering criterion to preserve more correct matches and shift the filtering task to visibility or geometry tools. Li et al introduce a RANSAC sampling strategy by prioritizing samples with frequent co-visibility [21].…”

“…Camposeco et al propose a geometric outlier filtering approach, in which a novel 2-point solver is able to compute an approximate camera position [6]. Assuming that the gravity direction and an approximate estimation of camera height are known, both Zeisl et al and Svarm et al present geometric outlier filtering approaches to handle extremely large outlier ratios [6,37]. Toft et al derive an outlier filtering method by combining the known gravity direction prior and semantic information [38].…”