Using Image Sequences for Long-Term Visual Localization

Abstract

Estimating the pose of a camera in a known scene, i.e., visual localization, is a core task for applications such as self-driving cars. In many scenarios, image sequences are available and existing work on combining single-image localization with odometry offers to unlock their potential for improving localization performance. Still, the largest part of the literature focuses on single-image localization and ignores the availability of sequence data. The goal of this paper is to demonstrate the potential of image sequences in challenging scenarios, e.g., under day-night or seasonal changes. Combining ideas from the literature, we describe a sequence-based localization pipeline that combines odometry with both a coarse and a fine localization module. Experiments on long-term localization datasets show that combining single-image global localization against a prebuilt map with a visual odometry / SLAM pipeline improves performance to a level where the extended CMU Seasons dataset can be considered solved. We show that SIFT features can perform on par with modern state-of-the-art features in our framework, despite being much weaker and a magnitude faster to compute. Our code is publicly available at github.com/rulllars.

Probabilistic Appearance-Invariant Topometric Localization With New Place Awareness

“…The mean distance traveled for our discrete filter at the 99% precision level is similar to MCL, meaning the similar localization performance is not at the expense of localization latency. Consistent with LCD experiments, incorporation of more odometry information yields better localization performance compared to 1-dof (Stenborg20 [9]) or no (Xu20Topo [10]) odometry.…”

“…2) Topological Bayes filter from Xu et al [10], which does not use odometry information. 3) Topometric localization from Stenborg et al [9], which uses forward from odometry motion only.…”

“…However, both methods utilize only 1D (forward) odometry information instead of full 3-dof odometry utilized by our proposed method. Like our proposed approach, [9] performs backward smoothing after forward filtering, however their method assumes no route deviations between query and reference. We show that this assumption has a substantial impact on localization performance when it does not hold.…”

“…Key to this system is a novel, principled technique for conditioning transition probabilities between discrete states with continuous 3-dof odometry. This extends existing discrete filters which use none [10], [17] to limited [9] odometry and provides an alternative to continuous state systems utilizing particle filters [10], [18].…”

“…techniques [1]- [6] to an image captured at the corresponding place, enabling persistent localization. Localization systems built around appearance-based topometric maps have become a key component of the 6-dof visual localization literature [7]- [9], where hierarchical localization approaches have demonstrated state-of-the-art performance. In such a hierarchy, the robot first localizes to a coarse, topometric map, and this coarse localization is used to initialize a structure-based pose refinement step.…”