LENS: Localization enhanced by NeRF synthesis

Abstract

Figure 1: Novel view synthesis with LENS: given a 3D scene defined by a set of images labeled with corresponding poses (green cameras), we train a NeRF and render novel views (depicted images) on pose queries distributed across the scene (red cameras). Synthetic and real images are gathered to train a camera pose regression model, which performs twice better when evaluated on test set images (blue cameras) compared to the same model trained only on real training samples.