Relative Pose Estimation and Fusion of Omnidirectional and Lidar Cameras

Tamás, Levente; Fröhlich, Róbert; Kató, Zoltán

doi:10.1007/978-3-319-16181-5_49

Cited by 14 publications

(25 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most state-of-the-art approaches handle the 3D-2D registration between a camera and a depth sensor by using special calibration targets [1], [2], [3]. Other semi-automatic methods extract human-selected 3D and 2D shapes from both sensors which are then aligned [12], [13]. The mentioned methods achieve excellent results and can therefore be used for a suitable initial calibration.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

RegNet: Multimodal sensor registration using deep neural networks

Schneider

Piewak

Stiller

et al. 2017

2017 IEEE Intelligent Vehicles Symposium (IV)

152

View full text Add to dashboard Cite

Abstract-In this paper, we present RegNet, the first deep convolutional neural network (CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between multimodal sensors, exemplified using a scanning LiDAR and a monocular camera. Compared to existing approaches, RegNet casts all three conventional calibration steps (feature extraction, feature matching and global regression) into a single real-time capable CNN. Our method does not require any human interaction and bridges the gap between classical offline and target-less online calibration approaches as it provides both a stable initial estimation as well as a continuous online correction of the extrinsic parameters. During training we randomly decalibrate our system in order to train RegNet to infer the correspondence between projected depth measurements and RGB image and finally regress the extrinsic calibration. Additionally, with an iterative execution of multiple CNNs, that are trained on different magnitudes of decalibration, our approach compares favorably to state-of-the-art methods in terms of a mean calibration error of 0.28• for the rotational and 6 cm for the translation components even for large decalibrations up to 1.5 m and 20• .

show abstract

Section: Related Workmentioning

confidence: 99%

“…The mentioned methods achieve excellent results and can therefore be used for a suitable initial calibration. However, they are either time consuming [12], [13], [2], [3]. or require a controlled environment [1].…”

Section: Related Workmentioning

confidence: 99%

RegNet: Multimodal sensor registration using deep neural networks

Schneider

Piewak

Stiller

et al. 2017

2017 IEEE Intelligent Vehicles Symposium (IV)

152

View full text Add to dashboard Cite

show abstract

“…Line features were introduced by L. Liu et al [4] to the registration of building data, as well. In yuanfang0524@gmail.com) addition, region features were segmented from 2D-3D urban sensing data and applied to registration by Levente Tamas et al [10]. In general, the extraction and matching of features are crucial to the success of feature-based 2D-3D registration methods.…”

Section: Introduction a Backgroundmentioning

confidence: 99%

2D-image to 3D-range registration in urban environments via scene categorization and combination of similarity measurements

Zhao

Wang

Tsai

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Statistical similarity measurements, such as mutual information (MI) and normalized mutual information (NMI), show potential in the registration of 2D-image to 3Drange scans collected in urban environments. However 2D-3D registration with these measurements are of limited usage in urban sensing applications because: 1) it relies on the diversity and dependency between pre-defined pair of 2D-3D attributes, such as the intensity from images and reflectivity from range scans, in the urban sensing environment, and 2) it requires high-end range sensors with strong abilities to capture reflectivity in urban scenarios. In this paper, we propose a robust way of estimating statistical similarity measurements for 2D-3D data that are collected in various urban scenes with both low-cost and high-end range sensors. Rather than estimate the similarity of 2D-3D data on specific pair of 2D-3D attributes, we compute similarity measurements between a set of 2D-3D attribute-pairs that could be dominant in the category of sensed urban scene and combine them into a reliable similarity measurement. By applying the combined similarity measurement to the common framework of statistical 2D-3D registration, we get superior results when compared with stateof-art similarity measurements (MI and NMI) in terms of registration accuracy and robustness to initial condition, as indicated by experiments conducted on two datasets that are collected in various urban scenes, with low-cost and high-end sensors.

show abstract

“…Furthermore, since correspondences are not available, (3.1) cannot be used directly. However, individual point matches can be integrated out yielding the following integral equation [Tamas, Frohlich, Kato, 2014]:…”

Section: Absolute Pose Of Spherical Camerasmentioning

confidence: 99%

“…To get an explicit formula for the above surface integrals, the spherical patches D S and F S can be naturally parametrized via Φ and Ψ over the planar regions D and F. Without loss of generality, we can assume that the third coordinate of X ∈ F is 0, hence D ⊂ R 2 , F ⊂ R 2 ; and ∀X S ∈ D S : X S = Φ(x), x ∈ D as well as ∀Z S ∈ F S : Z S = Ψ(X), X ∈ F yielding the following form of (3.2) [Tamas, Frohlich, Kato, 2014]:…”

Section: Absolute Pose Of Spherical Camerasmentioning

confidence: 99%

Region-Based Pose and Homography Estimation for Central Cameras

Fröhlich

View full text Add to dashboard Cite

I would like to express first and foremost my sincere gratitude to my advisor, Zoltán Kató, for the continuous support of my Ph.D study and related research. He has been a trustworthy guide both in my research and personal life. Thank you for noticing me and offering this possibility that truly changed my life. I would also like to thank my consultant, Levente Tamás, for his continuous support, good example, and all the insightful comments and encouragement. I would like to thank Iza for her support and patience during my work. I am grateful to my family for their support and for the opportunities they have created for me to achieve my goals. I am also thankful to my friends and colleagues in the RGVC group.

show abstract

Relative Pose Estimation and Fusion of Omnidirectional and Lidar Cameras

Cited by 14 publications

References 28 publications

RegNet: Multimodal sensor registration using deep neural networks

RegNet: Multimodal sensor registration using deep neural networks

2D-image to 3D-range registration in urban environments via scene categorization and combination of similarity measurements

Region-Based Pose and Homography Estimation for Central Cameras

Contact Info

Product

Resources

About