Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Chen, Shuai; Wang, Zirui; Prisacariu, Victor Adrian

doi:10.48550/arxiv.2104.04073

Cited by 3 publications

(6 citation statements)

References 62 publications

(109 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also report results from TransPoseNet [11], which is a state of the art transformer approach for camera pose regression. DirectPoseNet [16] As expected, we observe that using a higher grid resolution containing more samples leads to a better localization. In addition to that, we can see in figure 4 that the relative improvement compared to the baseline (i.e.…”

Section: Comparison With Related Localization Methodssupporting

confidence: 83%

“…iNeRF [15] performs gradient-based optimization to recover a pose estimate thanks to NeRF differentiability. Direct-PoseNet [16] adapts this idea to camera pose regression training by using an additional photometric loss between query image and NeRF synthesis on the predicted pose. PoseGan [17] learns jointly pose regression and view synthesis resulting in an improved localization accuracy.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

LENS: Localization enhanced by NeRF synthesis

Moreau,

Piasco,

Tsishkou

et al. 2021

Preprint

View full text Add to dashboard Cite

Figure 1: Novel view synthesis with LENS: given a 3D scene defined by a set of images labeled with corresponding poses (green cameras), we train a NeRF and render novel views (depicted images) on pose queries distributed across the scene (red cameras). Synthetic and real images are gathered to train a camera pose regression model, which performs twice better when evaluated on test set images (blue cameras) compared to the same model trained only on real training samples.

show abstract

Section: Comparison With Related Localization Methodssupporting

confidence: 83%

Section: Related Workmentioning

confidence: 99%

LENS: Localization enhanced by NeRF synthesis

Moreau,

Piasco,

Tsishkou

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…First, we evaluate the codebook, whose performance is the upper bound on what we can achieve with the full pipeline. We next compare our method to GQN-based methods [14,20,66] that also do not use continuous volumetric scene representations. We continue by evaluating our approach on other synthetic data.…”

Section: Methodsmentioning

confidence: 99%

“…Pose regression methods train a convolutional neural network (CNN) to regress the camera pose of an input image. There are two categories: absolute pose regression (APR) methods [5,8,14,28,30,33,41,60] and relative pose regression (RPR) methods [1,19,31,33,39]. It was shown [59] that APR is often not (much) more accurate than IR.…”

Section: Visual Localizationmentioning

confidence: 99%

“…While our approach performs relative pose regression, the main focus of our method is on the novel view synthesis. Some pose regression methods use novel view synthesis methods [14,41,42,44], however, they assume there is a method that generates images, whereas our method performs both the novel view synthesis and camera pose regression in a single model. Iterative refinement pose regression methods [57,70] start with an initial camera pose estimate and refine it by an iterative process, however, our approach generates novel views and the camera pose estimates in a single forward pass.…”

Section: Visual Localizationmentioning

confidence: 99%

See 1 more Smart Citation

ViewFormer: NeRF-Free Neural Rendering from Few Images Using Transformers

Kulhánek

Derner

Sattler

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Novel view synthesis is a long-standing problem. In this work, we consider a variant of the problem where we are given only a few context views sparsely covering a scene or an object. The goal is to predict novel viewpoints in the scene, which requires learning priors. The current state of the art is based on Neural Radiance Field (NeRF), and while achieving impressive results, the methods suffer from long training times as they require evaluating millions of 3D point samples via a neural network for each image. We propose a 2D-only method that maps multiple context views and a query pose to a new image in a single pass of a neural network. Our model uses a two-stage architecture consisting of a codebook and a transformer model. The codebook is used to embed individual images into a smaller latent space, and the transformer solves the view synthesis task in this more compact space. To train our model efficiently, we introduce a novel branching attention mechanism that allows us to use the same model not only for neural rendering but also for camera pose estimation. Experimental results on real-world scenes show that our approach is competitive compared to NeRF-based methods while not reasoning explicitly in 3D, and it is faster to train.

show abstract

Neural Fields in Visual Computing and Beyond

Xie¹,

Takikawa²,

Saito³

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in machine learning have increased interest in solving visual computing problems using a class of coordinate-based neural networks that parameterize physical properties of scenes or objects across space and time. These methods, which we call neural fields, have seen widespread success in applications such as 3D shape and image synthesis, animation of human bodies, 3D reconstruction, and pose estimation. Rapid progress has led to numerous papers, but a formulation and review of the problems in visual computing has not yet emerged. We provide context, mathematical grounding, and a review of 251 papers in the literature on neural fields. In Part I, we focus on neural fields techniques by identifying common components of neural field methods, including different generalization, representation, forward map, architecture, and manipulation methods. In Part II, we focus on applications of neural fields to different problems in visual computing, and beyond (e.g., robotics, audio). Our review shows the breadth of topics already covered in visual computing, both historically and in current incarnations, and highlights the improved quality, flexibility, and capability brought by neural field methods. Finally, we present a companion website that contributes a living database that can be continually updated by the community. CCS Concepts

show abstract

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Cited by 3 publications

References 62 publications

LENS: Localization enhanced by NeRF synthesis

LENS: Localization enhanced by NeRF synthesis

ViewFormer: NeRF-Free Neural Rendering from Few Images Using Transformers

Neural Fields in Visual Computing and Beyond

Contact Info

Product

Resources

About