RPNet: An End-to-End Network for Relative Camera Pose Estimation

Abstract

This paper addresses the task of relative camera pose estimation from raw image pixels, by means of deep neural networks. The proposed RPNet network takes pairs of images as input and directly infers the relative poses, without the need of camera intrinsic/extrinsic. While state-of-the-art systems based on SIFT + RANSAC, are able to recover the translation vector only up to scale, RPNet is trained to produce the full translation vector, in an end-to-end way. Experimental results on the Cambridge Landmark data set show very promising results regarding the recovery of the full translation vector. They also show that RPNet produces more accurate and more stable results than traditional approaches, especially for hard images (repetitive textures, textureless images, etc.). To the best of our knowledge, RPNet is the first attempt to recover full translation vectors in relative pose estimation.

Relative Camera Pose Estimation using Synthetic Data with Domain Adaptation via Cycle-Consistent Adversarial Networks

“…The result shows that the change interval of β in the outdoor scene is between 250 and 2000. Using cross-validation, RPNet [7] found the most suitable hyperparameter β value in different locations, and spends lots of time clustering the original dataset and testing the trained model for the evaluation. For RCPNet, we use automatic weights that scale on the loss function based on homoscedastic uncertainty (as in [55]) across all the locations, which is numerically more stable than β.…”

“…Different from RPNet [7] and PoseNet [11] based on GoogLeNet, we use two branches of pre-trained ResNet34 networks [57] to construct a weight-sharing Siamese network [56]. The 6DoF relative camera pose is estimated end-to-end.…”

“…An effective method is needed to produce image pairs to achieve relative camera pose estimation. For the Cambridge Landmarks dataset, En et al [7] randomly paired every image with eight images in the same sequence. The training sequences and testing sequences are separated beforehand.…”

“…The images are rescaled to 256 × n or n × 256 pixels, n ≥ 256, and then are cropped into 224 × 224 patches as the input of CNN in the previous work [7,11]. The model is trained by random cropping and then tested by central cropping as data augmentation.…”

“…RCPNet is first compared with other learning-based pose estimation approaches PoseNet [11,55] and RPNet [7], using the real images from two datasets, namely, Tuebingen Buildings and Cambridge Landmarks. We then compare the accuracy between the real images and synthetic images trained RCPNet models on the two datasets.…”