Privacy Preserving Image-Based Localization

Abstract

a) 3D Point Cloud (b) Inversion Attack on SIFT of 3D Point Cloud (c) 3D Line Cloud Projected 3D Points Reconstructed Image Original Image Figure 1: (a) Traditional image-based localization using 3D point cloud, which reveals potentially confidential information in the scene. (b) Reconstructed image using projected sparse 3D points and their SIFT features [50]. (c) Our proposed 3D line cloud protects user privacy by concealing the scene geometry and preventing inversion attacks, while still enabling accurate and efficient localization. AbstractImage-based localization is a core component of many augmented/mixed reality (AR/MR) and autonomous robotic systems. Current localization systems rely on the persistent storage of 3D point clouds of the scene to enable camera pose estimation, but such data reveals potentially sensitive scene information. This gives rise to significant privacy risks, especially as for many applications 3D mapping is a background process that the user might not be fully aware of. We pose the following question: How can we avoid disclosing confidential information about the captured 3D scene, and yet allow reliable camera pose estimation? This paper proposes the first solution to what we call privacy preserving image-based localization. The key idea of our approach is to lift the map representation from a 3D point cloud to a 3D line cloud. This novel representation obfuscates the underlying scene geometry while providing sufficient geometric constraints to enable robust and accurate 6-DOF camera pose estimation. Extensive experiments on several datasets and localization scenarios underline the high practical relevance of our proposed approach.

Revealing Scenes by Inverting Structure From Motion Reconstructions

“…Our work highlights the privacy and security risks associated with storing 3D point clouds and the necessity for developing privacy preserving point cloud representations and camera localization techniques, where the persistent scene model data cannot easily be inverted to reveal the appearance of the underlying scene. This was also the primary goal in concurrent work on privacy preserving camera pose estimation [41] which proposes a defense against the type of attacks investigated in our paper. Another interesting avenue of future work would be to explore privacy preserving features for recovering correspondences between images and 3D models.…”

“…To defend against CNN-based attacks, attempts at learning CNN-resistant transformations have shown some promise [33,10,35,13]. Concurrent to our work, Speciale et al [41] introduced the privacy preserving image-based localization problem to address the privacy issues we have brought up. They proposed a new camera pose estimation technique using an obfuscated representation of the map geometry which can defend against our inversion attack.…”

LensCap

“…The third app shares the AR point cloud, which contains the 3D visual corner points that are used to track the space. AR apps can share point clouds among users for shared positioning and/or send it to the cloud for object detection [13] and/or image-based localization [58]. The fourth app shares the face tracking result.…”

FloorVLoc: A Modular Approach to Floorplan Monocular Localization

“…Image-Based Localization (IBL) [9][10][11][12] has broadly been a popular approach to localization due to the amount of information that is provided by images. State-of-the-art algorithms for IBL have continued to follow a 3D structure-based approach [13][14][15][16][17][18][19][20], where 2D-3D image point to 3D world point correspondences are established for a query image, and then the camera pose is solved using an n-point-pose (PnP) solver.…”