SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

Zhu, Xinge; Ma, Yuexin; Wang, Tai; Xu, Yan; Shi, Jianping; Lin, Dahua

doi:10.1007/978-3-030-58595-2_35

Cited by 66 publications

(27 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PolarNet [3] introduced bird's-eye-view representation under the polar coordinates to balance the points across grid cells in a polar system. To maintain the 3D geometric relation of points, some methods [14,15,16,17,18] use the voxel partition and apply regular 3D convolutions for LiDAR segmentation. But their performances become limited for outdoor scenes due to the special properties of outdoor LiDAR point cloud mentioned above.…”

Section: Lidar Point Cloud Segmentationmentioning

confidence: 99%

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

Cong

Zhu

2021

Preprint

Self Cite

View full text Add to dashboard Cite

A thorough and holistic scene understanding is crucial for autonomous vehicles, where LiDAR semantic segmentation plays an indispensable role. However, most existing methods focus on the network design while neglecting the inherent difficulty, i.e, imbalanced data distribution in the realistic dataset (also named long-tailed distribution), which narrows down the capability of state-of-the-art methods. In this paper, we propose an input-output balanced framework to handle the issue of long-tailed distribution. Specifically, for the input space, we synthesize these tailed instances from mesh models and well simulate the position and density distribution of LiDAR scan, which enhances the input data balance and improves the data diversity. For the output space, a multihead block is proposed to group different categories based on their shapes and instance amounts, which alleviates the biased representation of dominating category during the feature learning. We evaluate the proposed model on two large-scale datasets, i.e, SemanticKITTI and nuScenes, where state-ofthe-art results demonstrate its effectiveness. The proposed new modules can also be used as a plug-and-play, and we apply them on various backbones and datasets, showing its good generalization ability.

show abstract

Section: Lidar Point Cloud Segmentationmentioning

confidence: 99%

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

Cong

Zhu

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These methods process LiDAR data in different representations. [8,17,44,45,35,46,47] utilize structured voxel representation to quantize the LiDAR data and feed them into 2D or 3D CNN to detect 3D object, while [4,22,15] project the LiDAR data into 2D bird's eye view or front view representations. Instead of transforming the representations of point cloud, [31,33] directly takes raw point cloud as input to localize 3D object based on the frustum region.…”

Section: Related Workmentioning

confidence: 99%

SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimation

Peng¹,

Zhu²,

Wang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

3D detection plays an indispensable role in environment perception. Due to the high cost of commonly used LiDAR sensor, stereo vision based 3D detection, as an economical yet effective setting, attracts more attention recently. For these approaches based on 2D images, accurate depth information is the key to achieve 3D detection, and most existing methods resort to a preliminary stage for depth estimation. They mainly focus on the global depth and neglect the property of depth information in this specific task, namely, sparsity and locality, where exactly accurate depth is only needed for these 3D bounding boxes. Motivated by this finding, we propose a stereo-image based anchor-free 3D detection method, called structure-aware stereo 3D detector (termed as SIDE), where we explore the instancelevel depth information via constructing the cost volume from RoIs of each object. Due to the information sparsity of local cost volume, we further introduce match reweighting and structure-aware attention, to make the depth information more concentrated. Experiments conducted on the KITTI dataset show that our method achieves the state-ofthe-art performance compared to existing methods without depth map supervision.

show abstract

“…In this experiment, we follow SECOND method and replace the regular voxelization and 3D convolution with the proposed cylindrical partition and asymmetrical 3D convolution networks, respectively. Similarly, to verify its scalability, we also extend the proposed modules to SSN [53]. The experiments are conducted on nuScenes dataset and the cylindrical partition also generates the 480 × 360 × 32 representation.…”

Section: Generalization Analysesmentioning

confidence: 99%

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Zhu¹,

Zhou²,

Wang³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

State-of-the-art methods for large-scale driving-scene LiDAR segmentation often project the point clouds to 2D space and then process them via 2D convolution. Although this corporation shows the competitiveness in the point cloud, it inevitably alters and abandons the 3D topology and geometric relations. A natural remedy is to utilize the 3D voxelization and 3D convolution network. However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited. An important reason is the property of the outdoor point cloud, namely sparsity and varying density. Motivated by this investigation, we propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. Moreover, a point-wise refinement module is introduced to alleviate the interference of lossy voxel-based label encoding. We evaluate the proposed model on two large-scale datasets, i.e., SemanticKITTI and nuScenes. Our method achieves the 1st place in the leaderboard of SemanticKITTI 1 and outperforms existing methods on nuScenes with a noticeable margin, about 4%. Furthermore, the proposed 3D framework also generalizes well to LiDAR panoptic segmentation and LiDAR 3D detection.

show abstract

SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

Cited by 66 publications

References 34 publications

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimation

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Contact Info

Product

Resources

About