Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis

Abstract

Visual Localization is one of the key enabling technologies for autonomous driving and augmented reality. High quality datasets with accurate 6 Degree-of-Freedom (DoF) reference poses are the foundation for benchmarking and improving existing methods. Traditionally, reference poses have been obtained via Structure-from-Motion (SfM). However, SfM itself relies on local features which are prone to fail when images were taken under different conditions, e.g., day/night changes. At the same time, manually annotating feature correspondences is not scalable and potentially inaccurate. In this work, we propose a semi-automated approach to generate reference poses based on feature matching between renderings of a 3D model and real images via learned features. Given an initial pose estimate, our approach iteratively refines the pose based on feature matches against a rendering of the model from the current pose estimate. We significantly improve the nighttime reference poses of the popular Aachen Day–Night dataset, showing that state-of-the-art visual localization methods perform better (up to 47%) than predicted by the original reference poses. We extend the dataset with new nighttime test images, provide uncertainty estimates for our new reference poses, and introduce a new evaluation criterion. We will make our reference poses and our framework publicly available upon publication.

Using Image Sequences for Long-Term Visual Localization

“…One possibility is that the ground truth poses are not precise enough for this regime. The facts that no methods listed on visuallocalization.net reaches 70% in the high precision regime for daytime queries, and that recently significantly improved poses for another dataset from [67] were released [90], support this idea.…”

Decoupling Makes Weakly Supervised Local Feature Better

“…However, our method significantly surpasses DELF under viewpoint change (10.553 vs. 3.797) and achieves much better overall performance (11.235 vs. 8.274). 2) Visual Localization: We then evaluate our method on the visual localization task with the Aachen Day-Night dataset [48]. For fair comparison, we adopt the official visual localization pipeline 1 used in the local feature challenge of workshop on long-term visual localization under changing conditions.…”

“…Finding pixel correspondences is a fundamental problem in computer vision. Sparse local feature [20], [5], [33], [11], as one of the mainstream methods to find correspondences, has been widely applied in many areas, such as simultaneous localization and mapping (SLAM) [27], [49], structure from motion (SfM) [37], [1], and visual localization [35], [48].…”

A Critical Analysis of Image-based Camera Pose Estimation Techniques

“…For example, images closer to the camera will intuitively get a smaller error range compared to further away images. Thus, sampled error thresholds could set the error thresholds per image [12] using a set of sampling k ratios, e.g., 50%, 30% and 10% respectively.…”