InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

Abstract

We seek to predict the 6 degree-of-freedom (6DoF) pose of a query photograph with respect to a large indoor 3D map. The contributions of this work are three-fold. First, we develop a new large-scale visual localization method targeted for indoor environments. The method proceeds along three steps: (i) efficient retrieval of candidate poses that ensures scalability to large-scale environments, (ii) pose estimation using dense matching rather than local features to deal with textureless indoor scenes, and (iii) pose verification by virtual view synthesis to cope with significant changes in viewpoint, scene layout, and occluders. Second, we collect a new dataset with reference 6DoF poses for large-scale indoor localization. Query photographs are captured by mobile phones at a different time than the reference 3D map, thus presenting a realistic indoor localization scenario. Third, we demonstrate that our method significantly outperforms current state-of-the-art indoor localization approaches on this new challenging data.

Is This the Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization

“…The geotag of the most similar database image is then often used to approximate the pose of the query image [30,31,76,83]. Place recognition approaches can also be used as part of a visual localization pipeline [13,29,53,62,72]: 2D-3D matching can be restricted to the parts of the scene visible in a short list of n visually similar database images, resulting in one pose estimate per retrieved image. This restriction helps to avoid global ambiguities in a scene, e.g., caused by similar structures found in unrelated parts of a scene, during matching [54].…”

“…In this paper, we follow this strategy. However, unlike previous work focused on improving the retrieval [1,4,20,26,34,36] or matching [56,72], we focus on the pose verification stage, i.e., the problem of selecting the "best" pose from the n estimated poses. An alternative to the localization approaches outlined above is to train a CNN that directly regresses the camera pose from a given input image [9,12,32,33,50,78].…”

“…View synthesis. Following [72], we use view synthesis to verify estimated camera poses by re-rendering the scene from the estimated viewpoints. View synthesis has also been used to enable localization under strong appearance [8,76] or viewpoint [64,66,76] changes.…”

“…State-of-the-art approaches for accurate visual localization are based on matches between 2D image and 3D scene coordinates [10,11,16,40,46,47,56,71,72,84]. These 2D-3D matches are either established using explicit feature matching [40,56,71,72,84] or via learning-based scene co- Figure 1: Using further modalities for indoor visual localization. Given a set of camera pose estimates for a query image (a, g), we seek to identify the most accurate estimate.…”

“…Given a set of camera pose estimates for a query image (a, g), we seek to identify the most accurate estimate. (b, h) Due to severe occlusion and weak textures, a stateof-the-art method [72] fails to identify the correct camera pose. To overcome those difficulties, we use several modalities along with visual appearance: (top) surface normals and (bottom) semantics.…”