TransLO: A Window-Based Masked Point Transformer Framework for Large-Scale LiDAR Odometry

Abstract

Recently, transformer architecture has gained great success in the computer vision community, such as image classification, object detection, etc. Nonetheless, its application for 3D vision remains to be explored, given that point cloud is inherently sparse, irregular, and unordered. Furthermore, existing point transformer frameworks usually feed raw point cloud of N×3 dimension into transformers, which limits the point processing scale because of their quadratic computational costs to the input size N. In this paper, we rethink the structure of point transformer. Instead of directly applying transformer to points, our network (TransLO) can process tens of thousands of points simultaneously by projecting points onto a 2D surface and then feeding them into a local transformer with linear complexity. Specifically, it is mainly composed of two components: Window-based Masked transformer with Self Attention (WMSA) to capture long-range dependencies; Masked Cross-Frame Attention (MCFA) to associate two frames and predict pose estimation. To deal with the sparsity issue of point cloud, we propose a binary mask to remove invalid and dynamic points. To our knowledge, this is the first transformer-based LiDAR odometry network. The experiment results on the KITTI odometry dataset show that our average rotation and translation RMSE achieves 0.500°/100m and 0.993% respectively. The performance of our network surpasses all recent learning-based methods and even outperforms LOAM on most evaluation sequences.Codes will be released on https://github.com/IRMVLab/TransLO.

3D LiDAR SLAM: A survey

“…The rapid evolution of 3D computer vision has partly propelled learning-based odometry frameworks (Chen, Wang, et al, 2021;Zhou et al, 2023), which primarily focus on feature extraction and matching rather than providing a complete solution. Despite yielding impressive results, enhancing their generalization capabilities (Chen et al, 2020;Li, Kong, Zhao, Li, et al, 2021) and managing their substantial computational demands (Liu et al, 2023) remains a priority.…”

“…LiDAR has gained significant research interest because of its capacity to offer rich 3D information, wide field of view (FOV), and rapid update rates. Some methods have employed point cloud registration (Ji & Singh, 2017;Shan & Englot, 2018;Wang, Wang, Chen, & Xie, 2021), image representation (Cho et al, 2020;Wang, Saputra, et al, 2019), transformer (Liu et al, 2023), semantic information (Li, Kong, Zhao, Li, et al, 2021), branch and bound theory (Hess et al, 2016), and multi-source data fusion (Lin & Zhang, 2022a;Zuo et al, 2019) to enhance LiDAR SLAM. Despite their reported performance improvements, LiDAR SLAM still confronts the following challenges:…”

“…Two representative projection-based methods. (a) and (b) are originally shown inLiu et al (2023) and, respectively. intensity images, tracks sparse visual features, and mitigates motion distortion using the Gaussian process Gauss-Newton (GPGN)(Tong et al, 2012) for continuous-time state estimation.…”

“…SLO(Guadagnino et al, 2022) utilizes SuperPoint(DeTone et al, 2018) on intensity images for feature selection and introduces a self-supervised online fine-tuning procedure for the feature extraction network, which doesn't rely on labels or ground truth poses. As illustrated in Figure7a, TransLO(Liu et al, 2023) designs the first transformer-based LiDAR odometry, which can process tens of thousands of points simultaneously by projecting points onto a 2D surface and then feeding them into a local transformer with linear complexity.…”