Deep Learning for 3D Point Clouds: A Survey

Guo, Yulan; Wang, Hanyun; Hu, Qingyong; Líu, Hao; Liu, Li; Bennamoun, Mohammed

doi:10.1109/tpami.2020.3005434

Cited by 1,320 publications

(748 citation statements)

References 259 publications

Supporting

Mentioning

590

Contrasting

Unclassified

Order By: Relevance

“…Several follow-up works focus on designing convolutions that can learn local features for a point cloud efficiently [15,29,20,42,39,47]. Please also refer to the technical report by Guo et al [13] for further summary of many deep learning techniques for 3D point clouds.…”

Section: Related Workmentioning

confidence: 99%

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

Zhang

Hua

Rosen

et al. 2019

2019 International Conference on 3D Vision (3DV)

136

144

View full text Add to dashboard Cite

Recent progresses in 3D deep learning has shown that it is possible to design special convolution operators to consume point cloud data. However, a typical drawback is that rotation invariance is often not guaranteed, resulting in networks that generalizes poorly to arbitrary rotations. In this paper, we introduce a novel convolution operator for point clouds that achieves rotation invariance. Our core idea is to use lowlevel rotation invariant geometric features such as distances and angles to design a convolution operator for point cloud learning. The well-known point ordering problem is also addressed by a binning approach seamlessly built into the convolution. This convolution operator then serves as the basic building block of a neural network that is robust to point clouds under 6-DoF transformations such as translation and rotation. Our experiment shows that our method performs with high accuracy in common scene understanding tasks such as object classification and segmentation. Compared to previous and concurrent works, most importantly, our method is able to generalize and achieve consistent results across different scenarios in which training and testing can contain arbitrary rotations. Our implementation is publicly available at our project page 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

Zhang

Hua

Rosen

et al. 2019

2019 International Conference on 3D Vision (3DV)

136

144

View full text Add to dashboard Cite

show abstract

“…Most methods cannot cope with or generate high-resolution models within a reasonable time. For instance, the TL-embedding network [45] was designed for a 20 3 voxel grid; 3DShapeNets [13] and VConv-DAE [44] were designed for a 24 3 voxel grid with 3 voxels padding in each direction; VoxNet [40], 3D-R2N2 [18], and ORION [50] were designed for a 32 3 voxel grid; 3D-GAN was designed to generate a 64 3 occupancy grid as a 3D shape representation. As the voxel resolution increases, the occupied voxels become sparser in the 3D space, which leads to more unnecessary computation.…”

Section: Sparse Voxel Representation (Octree)mentioning

confidence: 99%

“…Wang et al [65] proposed deep closest point (DCP), which extends the traditional iterative closest point (ICP) method [66], using a deep learning method to obtain the transformation parameters. Recently, Guo et al [3] presented a survey focusing on deep learning models for point clouds, which provides more details in this field.…”

Section: Observationsmentioning

confidence: 99%

See 1 more Smart Citation

A survey on deep geometry learning: From a representation perspective

et al. 2020

View full text Add to dashboard Cite

Researchers have achieved great success in dealing with 2D images using deep learning. In recent years, 3D computer vision and geometry deep learning have gained ever more attention. Many advanced techniques for 3D shapes have been proposed for different applications. Unlike 2D images, which can be uniformly represented by a regular grid of pixels, 3D shapes have various representations, such as depth images, multi-view images, voxels, point clouds, meshes, implicit surfaces, etc. The performance achieved in different applications largely depends on the representation used, and there is no unique representation that works well for all applications. Therefore, in this survey, we review recent developments in deep learning for 3D geometry from a representation perspective, summarizing the advantages and disadvantages of different representations for different applications. We also present existing datasets in these representations and further discuss future research directions.

show abstract

“…. 3D object tracking [9]) could accurately predict the shape, position, and angle of objects in 3D space. Autonomous vehicles can use estimated positions and angles to predict and plan their own behaviors and paths to avoid collisions and violations.…”

Section: Introductionmentioning

confidence: 99%

A Novel Object Re-Track Framework for 3D Point Clouds

Feng

Jiao

Zhu

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

3D point cloud data is an important data source for autonomous vehicles to perceive the surroundings. Achieving accurate object tracking of 3D point clouds has become a challenging task. In this paper, we propose a 3D object two-stage re-track framework directly utilizing point clouds as the input, without using the ground truth as the reference box. The framework consists of a coarse stage and a fine stage. By tracking back the previous frames and expanding the search space for each frame, we add the fine stage to re-track the lost objects of the coarse stage. Moreover, we design a dense AutoEncoder to enhance the discrimination in the latent space and improve shape completion performance, thus improving tracking performance. A Sample Update Strategy is also proposed to aggregate similar model shape samples in different frames, which improves the quality of the model shape. In terms of motion models for the proposed re-track framework, we further compare Kalman Filter with PointLSTM and do an extensive analysis. Finally, we test the re-track framework on the KITTI tracking dataset and outperform the public benchmark by 17.1%/15.5%

show abstract

Deep Learning for 3D Point Clouds: A Survey

Cited by 1,320 publications

References 259 publications

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

A survey on deep geometry learning: From a representation perspective

A Novel Object Re-Track Framework for 3D Point Clouds

Contact Info

Product

Resources

About