Do We Really Need Scene-specific Pose Encoders?

Abstract

Visual pose regression models estimate the camera pose from a query image with a single forward pass. Current models learn pose encoding from an image using deep convolutional networks which are trained per scene. The resulting encoding is typically passed to a multi-layer perceptron in order to regress the pose. In this work, we propose that scene-specific pose encoders are not required for pose regression and that encodings trained for visual similarity can be used instead. In order to test our hypothesis, we take a shallow architecture of several fully connected layers and train it with pre-computed encodings from a generic image retrieval model. We find that these encodings are not only sufficient to regress the camera pose, but that, when provided to a branching fully connected architecture, a trained model can achieve competitive results and even surpass current state-of-the-art pose regressors in some cases. Moreover, we show that for outdoor localization, the proposed architecture is the only pose regressor, to date, consistently localizing in under 2 meters and 5 degrees.

Paying Attention to Activation Maps in Camera Pose Regression

“…To the best of our knowledge, the paradigm of regressing the camera pose from the final output of a CNN backbone was adopted by all regressors to date [30]. Variations to the architecture focused on alternatives to the original proposed CNN backbone [20,21,38,31] and on deeper, branching architectures for the MLP head [38,21]. Other works tried to address overfitting by averaging over predictions from models with randomly dropped activations [14] or by reducing the dimensionality of the global image encoding with Long-Short-Term-Memory (LSTM) layers [36].…”

“…This formulation was adopted by many pose regressors, however it still requires manually tuning the parameters' initialization for different datasets [34]. In a recent work [31], the authors trained the model separately for position and orientation in order to reduce the need of additional parameters, while achieving comparable accuracy. Alternative representations for the orientation were also proposed to gain better balance and stability of the pose loss [38,5].…”

“…where s x and s q are learned parameters. Recently, Shavit and Ferens showed the advantage of separately learning each task on its own [31]. Here, we follow a combined approach, where we first train the entire model to minimize Eq.…”

“…The appealing 5 ms runtime and simplicity (a single component instead of a heavy pipeline) paved the way to a new research paradigm for camera pose estimation. Numerous methods, soon to follow, aimed at maintaining the low runtime and memory requirements, while improving the accuracy and generalization of the original method [16,14,15,20,21,36,38,31,37,6].…”

“…Different camera pose regressors suggested different backbones [20,31], loss formulations [15] and MLP architectures [38,21] as well as additional manipulations of the output [36,37]. Common to all these methods, is that they apply the pose regression using a single global image encoding computed by the backbone CNN.…”