Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Zhu, Xinge; Zhou, Hui; Wang, Tai; Hong, Fangzhou; Ma, Yuexin; Wei, Li; Li, Hongsheng; Lin, Dahua

doi:10.1109/cvpr46437.2021.00981

Cited by 327 publications

(225 citation statements)

References 33 publications

Supporting

Mentioning

225

Contrasting

Order By: Relevance

“…As the sparse convolution is proposed in (Graham, 2014;Yan et al, 2018;Retinskiy, 2019), recent voxel-based methods (Yan et al, 2020;Tang et al, 2020;Zhu et al, 2020) accelerates the inference speed by removing the useless computations from empty voxels in point cloud, and also accomplishes successful semantic segmentation performance. However, those models are still not adequate in real-time performance due to the high voxel-resolution.…”

Section: Voxel-based Methodsmentioning

confidence: 99%

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

Park¹,

Kim²,

Jo³

2022

Preprint

View full text Add to dashboard Cite

The autonomous car must recognize the driving environment quickly for safe driving. As the Light Detection And Range (LiDAR) sensor is widely used in the autonomous car, fast semantic segmentation of LiDAR point cloud, which is the point-wise classification of the point cloud within the sensor framerate, has attracted attention in recognition of the driving environment. Although the voxel and fusion-based semantic segmentation models are the state-of-the-art model in point cloud semantic segmentation recently, their real-time performance suffer from high computational load due to high voxel resolution. In this paper, we propose the fast voxel-based semantic segmentation model using Point Convolution and 3D Sparse Convolution (PCSCNet). The proposed model is designed to outperform at both high and low voxel resolution using point convolution-based feature extraction. Moreover, the proposed model accelerates the feature propagation using 3D sparse convolution after the feature extraction. The experimental results demonstrate that the proposed model outperforms the state-of-the-art real-time models in semantic segmentation of SemanticKITTI and nuScenes, and achieves the real-time performance in LiDAR point cloud inference.

show abstract

Section: Voxel-based Methodsmentioning

confidence: 99%

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

Park¹,

Kim²,

Jo³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Voxelization based methods [10,11,12,13] transform the irregular unordered point cloud into regular 3D grids, and then the powerful 3D convolution is applied in feature extraction and prediction. However, the problem of granular information loss can be caused by using a large voxel size.…”

Section: Voxelization Based Methodsmentioning

confidence: 99%

MVP-Net: Multiple View Pointwise Semantic Segmentation of Large-Scale Point Clouds

Luo¹,

Li²,

Cheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Semantic segmentation of 3D point cloud is an essential task for autonomous driving environment perception. The pipeline of most pointwise point cloud semantic segmentation methods includes points sampling, neighbor searching, feature aggregation, and classification. Neighbor searching method like K-nearest neighbors algorithm, KNN, has been widely applied. However, the complexity of KNN is always a bottleneck of efficiency. In this paper, we propose an end-to-end neural architecture, Multiple View Pointwise Net, MVP-Net, to efficiently and directly infer large-scale outdoor point cloud without KNN or any complex pre/postprocessing. Instead, assumption-based sorting and multi-rotation of point cloud methods are introduced to point feature aggregation and receptive field expanding. Numerical experiments show that the proposed MVP-Net is 11 times faster than the most efficient pointwise semantic segmentation method RandLA-Net [1] and achieves the same accuracy on the large-scale benchmark SemanticKITTI dataset. KeywordsPoint Cloud • Semantic Segmentation • Autonomous Driving Recently, PointNet-based works [4, 1] were proposed to directly process large-scale point clouds. These pipelines include multi-level point cloud sampling, neighbor searching, and PointNet-based local feature aggregation. RandLA-

show abstract

“…But their performances become limited for outdoor scenes due to the special properties of outdoor LiDAR point cloud mentioned above. Then, focusing on the characteristics of sparsity and varying density, [4] proposed a cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation and achieved state-of-the-art performance. However, all these methods still leave the long-tailed problems, where the accuracy of rare categories are no less unsatisfying.…”

Section: Lidar Point Cloud Segmentationmentioning

confidence: 99%

“…Ablation studies are also conducted to validate each component of our approach and its generalization ability. Evaluation Metric To evaluate the results of our model, we follow previous work [4,3] and use mean intersection-overunion (mIoU) over all classes as the evaluation metric, which is given by mIoU = 1 C C i=1 |Pi∩Gi| |Pi∪Gi| , where P i and G i denote the predicted and labelled set of points for class i, and C is the number of total categories.…”

Section: Output Balanced Modulementioning

confidence: 99%

“…LiDAR semantic segmentation that could provide a thorough scene understanding, has attracted extensive studies. Most existing works usually pay much attention on point-to-structure representations, including spherical projection [1,2], bird-eye-view projection [3] and 3D voxelization [4,5], and another group of methods focus on the network architecture design [6,4,7]. However, these methods often neglect the inherit difficulty caused by the long-tailed data distribution.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

Cong

Zhu

2021

Preprint

Self Cite

View full text Add to dashboard Cite

A thorough and holistic scene understanding is crucial for autonomous vehicles, where LiDAR semantic segmentation plays an indispensable role. However, most existing methods focus on the network design while neglecting the inherent difficulty, i.e, imbalanced data distribution in the realistic dataset (also named long-tailed distribution), which narrows down the capability of state-of-the-art methods. In this paper, we propose an input-output balanced framework to handle the issue of long-tailed distribution. Specifically, for the input space, we synthesize these tailed instances from mesh models and well simulate the position and density distribution of LiDAR scan, which enhances the input data balance and improves the data diversity. For the output space, a multihead block is proposed to group different categories based on their shapes and instance amounts, which alleviates the biased representation of dominating category during the feature learning. We evaluate the proposed model on two large-scale datasets, i.e, SemanticKITTI and nuScenes, where state-ofthe-art results demonstrate its effectiveness. The proposed new modules can also be used as a plug-and-play, and we apply them on various backbones and datasets, showing its good generalization ability.

show abstract

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Cited by 327 publications

References 33 publications

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

MVP-Net: Multiple View Pointwise Semantic Segmentation of Large-Scale Point Clouds

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

Contact Info

Product

Resources

About