Spherical Interpolated Convolutional Network With Distance–Feature Density for 3-D Semantic Segmentation of Point Clouds

Wang, Guangming; Yang, Yehui; Zhang, Huixin; Liu, Zhe; Wang, Hesheng

doi:10.1109/tcyb.2021.3124954

Cited by 16 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since deep learning has shown excellent performance for raw point-cloud-based tasks [40,46,21,51] with the proposal of PointNet [29] and PointNet++ [30], many works estimate scene flow directly from raw point clouds in an end-to-end fashion. FlowNet3D [19] presents the first end-to-end scene flow estimation framework on point clouds.…”

Section: Related Workmentioning

confidence: 99%

What Matters for 3D Scene Flow Network

Wang¹,

Hu²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

3D scene flow estimation from point clouds is a low-level 3D motion perception task in computer vision. Flow embedding is a commonly used technique in scene flow estimation, and it encodes the point motion between two consecutive frames. Thus, it is critical for the flow embeddings to capture the correct overall direction of the motion. However, previous works only search locally to determine a soft correspondence, ignoring the distant points that turn out to be the actual matching ones. In addition, the estimated correspondence is usually from the forward direction of the adjacent point clouds, and may not be consistent with the estimated correspondence acquired from the backward direction. To tackle these problems, we propose a novel all-to-all flow embedding layer with backward reliability validation during the initial scene flow estimation. Besides, we investigate and compare several design choices in key components of the 3D scene flow network, including the point similarity calculation, input elements of predictor, and predictor & refinement level design. After carefully choosing the most effective designs, we are able to present a model that achieves the stateof-the-art performance on FlyingThings3D and KITTI Scene Flow datasets. Our proposed model surpasses all existing methods by at least 38.2% on FlyingTh-ings3D dataset and 24.7% on KITTI Scene Flow dataset for EPE3D metric. We release our codes at https://github.com/IRMVLab/3DFlow.

show abstract

Section: Related Workmentioning

confidence: 99%

What Matters for 3D Scene Flow Network

Wang¹,

Hu²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Latest works [1]- [4] show the potential of directly consuming points, and they do not need to convert the point clouds into other forms, such as voxel form [5]- [8]. Many works have explored the learning of single point cloud on 3D object retrieval [9]- [11], classification [1], [2], and segmentation [12]- [14]. There are a few pieces of research on the learning of multi-frame point cloud, and there remain some challenges.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we focus on a novel sparse 3D convolution method for dynamic 3D point cloud sequences based on direct grouping, not relying on extra scene flow estimation [4]. Inspired by the recent interpolated convolution methods [12], [14] on a single frame of 3D point clouds, we design an anchor-based spatial-temporal attention convolution to deal with dynamic 3D point cloud sequences. The results demonstrate that our structured spatial-temporal attention con-volution design benefits a lot to the tasks for 3D point cloud sequences.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Anchor-Based Spatio-Temporal Attention 3D Convolutional Networks for Dynamic 3D Point Cloud Sequences

Wang,

Chen,

Liu

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, learning based methods for the robot perception from the image or video have much developed, but deep learning methods for dynamic 3D point cloud sequences are underexplored. With the widespread application of 3D sensors such as LiDAR and depth camera, efficient and accurate perception of the 3D environment from 3D sequence data is pivotal to autonomous driving and service robots. An Anchor-based Spatial-Temporal Attention Convolution operation (ASTAConv) is proposed in this paper to process dynamic 3D point cloud sequences. The proposed convolution operation builds a regular receptive field around each point by setting several virtual anchors around each point. The features of neighborhood points are firstly aggregated to each anchor based on spatial-temporal attention mechanism. Then, anchor-based sparse 3D convolution is adopted to aggregate the features of these anchors to the core points. The proposed method makes better use of the structured information within the local region, and learn spatial-temporal embedding features from dynamic 3D point cloud sequences. Then Anchor-based Spatial-Temporal Attention Convolutional Neural Networks (ASTACNNs) are proposed for classification and segmentation tasks and are evaluated on action recognition and semantic segmentation tasks. The experimental results on MSRAction3D and Synthia datasets demonstrate that the higher accuracy can be achieved than the previous state-of-the-art method by our novel strategy of multi-frame fusion.

show abstract

What Matters for 3D Scene Flow Network

Wang

Liu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Spherical Interpolated Convolutional Network With Distance–Feature Density for 3-D Semantic Segmentation of Point Clouds

Cited by 16 publications

References 51 publications

What Matters for 3D Scene Flow Network

What Matters for 3D Scene Flow Network

Anchor-Based Spatio-Temporal Attention 3D Convolutional Networks for Dynamic 3D Point Cloud Sequences

What Matters for 3D Scene Flow Network

Contact Info

Product

Resources

About