PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths

Wen, Xin; Peng, Xiang; Han, Zhizhong; Cao, Yan‐Pei; Wan, Pengfei; Zheng, Wen; Liu, Yu‐Shen

doi:10.1109/cvpr46437.2021.00736

Cited by 137 publications

(100 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The PMP-Net++ proposed in this paper is an enhanced extension of our latest work PMP-Net [14]. We find that the point features learned in the moving procedure plays the key role during the prediction of high quality complete shape.…”

Section: Introductionmentioning

confidence: 89%

“…At step k, in order to predict the displacement vector ∆p k i for each point, we first extract per-point features from the point cloud. In the previous implementation of our PMP-Net [14], this is achieved by first adopting the basic framework of PointNet++ [15] to extract the global feature of input the 3D shape, and then using the feature propagation module to propagate the global feature to each point in the 3D space, and finally producing per-point feature h k,l i for point p k i . In PMP-Net++, we adopt the recent implementation success of transformer [49] to enhance the point feature learned by the PointNet++, where we follow the practice of Point Transformer [16], and add an additional transformer module between each set abstraction (SA) layer of PointNet++ based encoder.…”

Section: Transformer-enhanced Displacement Predictionmentioning

confidence: 99%

“…In this subsection, we analyze the influence of different parts in the PMP-Net++, and compared it with PMP-Net to analysis the effectiveness of the incremental contribution. By default, we use the same network sittings in PMP-Net [14] for all experiments, where all studies are typically conducted on the validation set of Completion3D dataset under four categories (i.e. plane, car, chair and table) for convenience.…”

Section: Model Analysismentioning

confidence: 99%

See 2 more Smart Citations

PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Paths

Xu¹,

Peng²,

Han³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Point cloud completion concerns to predict missing part for incomplete 3D shapes. A common strategy is to generate complete shape according to incomplete input. However, unordered nature of point clouds will degrade generation of high-quality 3D shapes, as detailed topology and structure of unordered points are hard to be captured during the generative process using an extracted latent code. We address this problem by formulating completion as point cloud deformation process. Specifically, we design a novel neural network, named PMP-Net++, to mimic behavior of an earth mover. It moves each point of incomplete input to obtain a complete point cloud, where total distance of point moving paths (PMPs) should be the shortest. Therefore, PMP-Net++ predicts unique PMP for each point according to constraint of point moving distances. The network learns a strict and unique correspondence on point-level, and thus improves quality of predicted complete shape. Moreover, since moving points heavily relies on per-point features learned by network, we further introduce a transformer-enhanced representation learning network, which significantly improves completion performance of PMP-Net++. We conduct comprehensive experiments in shape completion, and further explore application on point cloud up-sampling, which demonstrate non-trivial improvement of PMP-Net++ over state-of-the-art point cloud completion/up-sampling methods.

show abstract

Section: Introductionmentioning

confidence: 89%

Section: Transformer-enhanced Displacement Predictionmentioning

confidence: 99%

Section: Model Analysismentioning

confidence: 99%

See 1 more Smart Citation

PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Paths

Xu¹,

Peng²,

Han³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We leverage Chamfer distance (CD) as the primary loss function. The L 2 version of Chamfer distance (CD) is defined as (6) where X and Y are point sets, x ∈ X and y ∈ Y are point coordinates, respectively. The L 1 version of CD replaces L 2 -norm in Eq.…”

Section: Completion Lossmentioning

confidence: 99%

“…I N 3D computer vision [1], [2], [3], [4] applications, raw point clouds captured by 3D scanners and depth cameras are usually sparse and incomplete [5], [6], [7] due to occlusion and limited sensor resolution. Therefore, point cloud completion [5], [8], which aims to predict a complete shape from its partial observation, is vital for various downstream tasks.…”

Section: Introductionmentioning

confidence: 99%

Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer

Peng¹,

Xu²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Most existing point cloud completion methods suffered from discrete nature of point clouds and unstructured prediction of points in local regions, which makes it hard to reveal fine local geometric details. To resolve this issue, we propose SnowflakeNet with Snowflake Point Deconvolution (SPD) to generate the complete point clouds. SPD models the generation of complete point clouds as the snowflake-like growth of points, where the child points are progressively generated by splitting their parent points after each SPD. Our insight of revealing detailed geometry is to introduce skip-transformer in SPD to learn point splitting patterns which can fit local regions the best. Skip-transformer leverages attention mechanism to summarize the splitting patterns used in previous SPD layer to produce the splitting in current SPD layer. The locally compact and structured point clouds generated by SPD precisely reveal the structure characteristic of 3D shape in local patches, which enables us to predict highly detailed geometries. Moreover, since SPD is a general operation, which is not limited to completion, we further explore the applications of SPD on other generative tasks, including point cloud auto-encoding, generation, single image reconstruction and upsampling. Our experimental results outperform the state-of-the-art methods under widely used benchmarks.

show abstract