INeRF: Inverting Neural Radiance Fields for Pose Estimation

Lin, Yiguang; Florence, Pete; Barron, Jonathan T.; Rodríguez, Alberto; Isola, Phillip; Lin, Tsung-Yi

doi:10.48550/arxiv.2012.05877

Cited by 22 publications

(27 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, we incorporate NeRF architecture to provide photometric supervision to the pose regression model. We consider two other works related to our paper: iNeRF [65] can estimate pose iteratively by inverting a pre-trained NeRF model on the test images. Wang et al [62] show that NeRF can jointly optimize camera parameters and 3D scene representation from RGB images alone.…”

Section: Related Workmentioning

confidence: 99%

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Chen¹,

Wang²,

Prisacariu³

2021

Preprint

View full text Add to dashboard Cite

We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we modify the rotation representation from the classical quaternion to SO(3) in pose regression, removing the need for balancing rotation and translation loss terms. As a result, our network Direct-PoseNet achieves state-of-the-art performance among all other single-image APR methods on the 7-Scenes benchmark and the LLFF dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Chen¹,

Wang²,

Prisacariu³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A few recent papers [1,3,8,23,44] attempt to predict scene-level geometry with RGB-(D) inputs, but they all assume given camera poses. Another set of works [17,51,59] tackle the problem of camera pose optimization, but they need a rather long optimization process, which is not suitable for real-time applications.…”

Section: Related Workmentioning

confidence: 99%

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

Zhu¹,

Peng²,

Larsson³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM). Nevertheless, existing methods produce oversmoothed scene reconstructions and have difficulty scaling up to large scenes. These limitations are mainly due to their simple fully-connected network architecture that does not incorporate local information in the observations. In this paper, we present NICE-SLAM, a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation. Optimizing this representation with pre-trained geometric priors enables detailed reconstruction on large indoor scenes. Compared to recent neural implicit SLAM systems, our approach is more scalable, efficient, and robust. Experiments on five challenging datasets demonstrate competitive results of NICE-SLAM in both mapping and tracking quality. Project page: https:// pengsongyou.github.io/ nice-slam.

show abstract

“…Instance-level 6D object pose estimation only estimates the 6D pose of a particular object can be divided into five parts: direct-methods [17,47,53], keypoint-based methods [29,34,37], dense coordinate-based methods [22,27,51], refinement based methods [21,25,48] and self-supervised methods [33,42]. There are also many methods propose to utilize RGBD data as input for instance-level object pose estimation [12,13,31,46].…”

Section: D Object Pose Estimationmentioning

confidence: 99%

ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation

Fan

Song

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, category-level 6D object pose estimation has achieved significant improvements with the development of reconstructing canonical 3D representations. However, the reconstruction quality of existing methods is still far from excellent. In this paper, we propose a novel Adversarial Canonical Representation Reconstruction Network named ACR-Pose. ACR-Pose consists of a Reconstructor and a Discriminator. The Reconstructor is primarily composed of two novel sub-modules: Pose-Irrelevant Module (PIM) and Relational Reconstruction Module (RRM). PIM tends to learn canonical-related features to make the Reconstructor insensitive to rotation and translation, while RRM explores essential relational information between different input modalities to generate high-quality features. Subsequently, a Discriminator is employed to guide the Reconstructor to generate realistic canonical representations. The Reconstructor and the Discriminator learn to optimize through adversarial training. Experimental results on the prevalent NOCS-CAMERA and NOCS-REAL datasets demonstrate that our method achieves state-of-the-art performance.

show abstract

INeRF: Inverting Neural Radiance Fields for Pose Estimation

Cited by 22 publications

References 34 publications

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation

Contact Info

Product

Resources

About