Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Abstract

State-of-the-art methods for large-scale driving-scene LiDAR segmentation often project the point clouds to 2D space and then process them via 2D convolution. Although this corporation shows the competitiveness in the point cloud, it inevitably alters and abandons the 3D topology and geometric relations. A natural remedy is to utilize the 3D voxelization and 3D convolution network. However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited. An important reason is the property of the outdoor point cloud, namely sparsity and varying density. Motivated by this investigation, we propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. Moreover, a point-wise refinement module is introduced to alleviate the interference of lossy voxel-based label encoding. We evaluate the proposed model on two large-scale datasets, i.e., SemanticKITTI and nuScenes. Our method achieves the 1st place in the leaderboard of SemanticKITTI 1 and outperforms existing methods on nuScenes with a noticeable margin, about 4%. Furthermore, the proposed 3D framework also generalizes well to LiDAR panoptic segmentation and LiDAR 3D detection.

Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation

“…Some studies [29,44] analyse the bottleneck of point-based and voxelbased methods and propose point-voxel convolution that combines both of them. [45,56] proposed customized 3D convolution networks and achieved impressive performance.…”

“…To address this issue, we design PCT-Net, a point cloud translation network that translates synthetic point clouds to have similar features as real point clouds. We disentangle synthetic-to-real gaps into an appearance component and a sparsity component [56] that encode the distribution of scene geometry and point sparsity, respectively. PCT-Net has an appearance translation module and a sparsity translation module that handle the two components as shown in Fig.…”

FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding

“…However, the improvement of those methods in the outdoor LiDAR point cloud remains limited. Some recent papers are trying to consider a better 3D voxelization method to cut the 3D space by incorporating how LiDAR sensor generate the point cloud [20].…”

Perception-Aware Multi-Sensor Fusion for 3D LiDAR Semantic Segmentation

“…The extensive experiments on two benchmark data sets demonstrate the superior performance of our method. For example, on nuScenes [7], PMF outperforms Cylinder3D [59], a state-of-the-art LiDARbased method, by 0.8% in mIoU.…”