Monocular camera localization in 3D LiDAR maps

Caselitz, Tim; Steder, Bastian; Ruhnke, Michael; Burgard, Wolfram

doi:10.1109/iros.2016.7759304

Cited by 134 publications

(84 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, in order to compare the localization performances with the state-of-the-art monocular localization in LiDAR maps [3], we calculated mean and standard deviation for both rotation and translation components over 10 runs on the sequence 00 of the KITTI odometry dataset. Our approach shows comparable values for the translation component (0.33 ± 0.22m w.r.t.…”

Section: Iterative Refinement and Overall Assessmentmentioning

confidence: 99%

“…Different options have been investigated to solve the localization problem, including approaches based on both vision and Light Detection And Ranging (LiDAR); they share the exploitation of an a-priori knowledge of the environment in the localization process [3]- [5]. Localization approaches that utilize the same sensor for mapping and localization usually achieve good performances, as the map of the scene is matched to the same kind of data generated by the onboard sensor.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CMRNet: Camera to LiDAR-Map Registration

Cattaneo¹,

Vaghi²,

Ballardini³

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network (CNN) to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i.e., CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07 • median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17 • . To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.

show abstract

Section: Iterative Refinement and Overall Assessmentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

CMRNet: Camera to LiDAR-Map Registration

Cattaneo¹,

Vaghi²,

Ballardini³

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

show abstract

“…Moreover, the authors sought the maximum normalized mutual information between real camera measurements and these synthetic views. Caselitz et al tracked a monocular camera within a LiDAR map by matching sparse camera point clouds acquired from ORB‐SLAM with an a priori LiDAR map to find corresponding points. However, the point cloud constructed by a camera is sparse and lacks structural feature information.…”

Section: Related Workmentioning

confidence: 99%

Scale‐aware camera localization in 3D LiDAR maps with a monocular visual odometry

Sun

Yang

Liu

2019

Computer Animation & Virtual

View full text Add to dashboard Cite

Localization information is essential for mobile robot systems in navigation tasks. Many visual‐based approaches focus on localizing a robot within prior maps acquired with cameras. It is critical where the Global Positioning System signal is unreliable. In contrast to conventional methods that localize a camera in an image‐based map, we propose a novel approach that localizes a monocular camera within a given three‐dimensional (3D) light detection and ranging (LiDAR) map. We employ visual odometry to reconstruct a semidense set of 3D points from the monocular camera images. These points are continuously matched against the 3D prior LiDAR map by a modified feature‐based point cloud registration method to track a full six‐degree‐of‐freedom camera pose. Since the monocular camera suffers from the scale‐drift problem due to the lack of depth information, the proposed method solves it by adopting updatable scale estimation. Experiments carried out on a publicly large‐scale data set demonstrate that the camera and LiDAR multimodal data matching problem is solved, and the localization accuracy of our method is comparable to state‐of‐the‐art approaches.

show abstract

“…For an accurate geo-localization not affected by scale drift ,prior information in a geographic information system (GIS) has been utilized in previous studies. For example, point clouds, 3D models, building footprints, and road maps arXiv:1808.08544v1 [cs.CV] 26 Aug 2018 have been proven to be efficient for correcting reconstructed 3D maps [5,6,7,8,9]. However, these priors are only available in limited situations, e.g., in an area that is observed in advance, or in an environment consisting of simply-shaped buildings.…”

Section: Introductionmentioning

confidence: 99%

Scale Drift Correction of Camera Geo-Localization Using Geo-Tagged Images

Iwami

Ikehata

Aizawa

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Camera geo-localization from a monocular video is a fundamental task for video analysis and autonomous navigation. Although 3D reconstruction is a key technique to obtain camera poses, monocular 3D reconstruction in a large environment tends to result in the accumulation of errors in rotation, translation, and especially in scale: a problem known as scale drift. To overcome these errors, we propose a novel framework that integrates incremental structure from motion (SfM) and a scale drift correction method utilizing geo-tagged images, such as those provided by Google Street View. Our correction method begins by obtaining sparse 6-DoF correspondences between the reconstructed 3D map coordinate system and the world coordinate system, by using geo-tagged images. Then, it corrects scale drift by applying pose graph optimization over Sim(3) constraints and bundle adjustment. Experimental evaluations on large-scale datasets show that the proposed framework not only sufficiently corrects scale drift, but also achieves accurate geo-localization in a kilometer-scale environment.

show abstract

Monocular camera localization in 3D LiDAR maps

Cited by 134 publications

References 21 publications

CMRNet: Camera to LiDAR-Map Registration

CMRNet: Camera to LiDAR-Map Registration

Scale‐aware camera localization in 3D LiDAR maps with a monocular visual odometry

Scale Drift Correction of Camera Geo-Localization Using Geo-Tagged Images

Contact Info

Product

Resources

About