Paying Attention to Activation Maps in Camera Pose Regression

Shavit, Yoli; Ferens, Ron; Keller, Yosi

doi:10.48550/arxiv.2103.11477

Cited by 1 publication

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regression-based localization trains a neural network and takes the network parameters as a global map representation which can directly regress the 6-DOF camera poses (Kendall, Grimes, and Cipolla 2015;Balntas, Li, and Prisacariu 2018;Kendall and Cipolla 2017;Moreau et al 2022a;Shavit, Ferens, and Keller 2021) or the 3D scene coordinate of each pixel (Cavallari et al 2017;Li et al 2020;Yang et al 2019) by taking the query image as the network input. For the simplicity and end-to-end training manner, these methods have attracted considerable attention.…”

Section: Introductionmentioning

confidence: 99%

PNeRFLoc: Visual Localization with Point-Based Neural Radiance Fields

Zhao,

Yang,

Mao

et al. 2024

AAAI

View full text Add to dashboard Cite

Due to the ability to synthesize high-quality novel views, Neural Radiance Fields (NeRF) has been recently exploited to improve visual localization in a known environment. However, the existing methods mostly utilize NeRF for data augmentation to improve the regression model training, and their performances on novel viewpoints and appearances are still limited due to the lack of geometric constraints. In this paper, we propose a novel visual localization framework, i.e., PNeRFLoc, based on a unified point-based representation. On one hand, PNeRFLoc supports the initial pose estimation by matching 2D and 3D feature points as traditional structure-based methods; on the other hand, it also enables pose refinement with novel view synthesis using rendering-based optimization. Specifically, we propose a novel feature adaption module to close the gaps between the features for visual localization and neural rendering. To improve the efficacy and efficiency of neural rendering-based optimization, we also developed an efficient rendering-based framework with a warping loss function. Extensive experiments demonstrate that PNeRFLoc performs the best on the synthetic dataset when the 3D NeRF model can be well learned, and significantly outperforms all the NeRF-boosted localization methods with on-par SOTA performance on the real-world benchmark localization datasets. Project webpage: https://zju3dv.github.io/PNeRFLoc/.

show abstract