PointProNets: Consolidation of Point Clouds with Convolutional Neural Networks

Abstract

With the widespread use of 3D acquisition devices, there is an increasing need of consolidating captured noisy and sparse point cloud data for accurate representation of the underlying structures. There are numerous algorithms that rely on a variety of assumptions such as local smoothness to tackle this ill‐posed problem. However, such priors lead to loss of important features and geometric detail. Instead, we propose a novel data‐driven approach for point cloud consolidation via a convolutional neural network based technique. Our method takes a sparse and noisy point cloud as input, and produces a dense point cloud accurately representing the underlying surface by resolving ambiguities in geometry. The resulting point set can then be used to reconstruct accurate manifold surfaces and estimate surface properties. To achieve this, we propose a generative neural network architecture that can input and output point clouds, unlocking a powerful set of tools from the deep learning literature. We use this architecture to apply convolutional neural networks to local patches of geometry for high quality and efficient point cloud consolidation. This results in significantly more accurate surfaces, as we illustrate with a diversity of examples and comparisons to the state‐of‐the‐art.

PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

“…Moreover, we found that a straightforward implementation might not converge, while with the proper loss and with an inflation term, the network can both stabilize and achieve higher accuracy. [RÖPG18]. The two plots on the left are evaluated on our test set without outliers, the two following plots compare the outlier removal performance using the f1 and the f2 scores and the right-most plot shows the denoising performance after outlier removal.…”

Differentiable Manifold Reconstruction for Point Cloud Denoising

“…In addition to denoising networks, some other neural network architectures involve point cloud consolidation, which includes denoising but is often only applicable to trivial noise. PointProNet [27] projects patches in the point cloud into 2D height maps and leverages a 2D CNN to denoise and upsample them. EC-Net [37] and 3PU [36] mainly focus on upsampling, and have shown to be robust against trivial noise.…”

A Network Architecture for Point Cloud Classification via Automatic Depth Images Generation

“…A key component of our method is a differentiable module to generate depth images from the points, which can be inserted into any neural network architecture. We took [21] where the authors create depth field images by projecting the points onto an image plane. However, their method is designed to handle local patches of points, where the geometry is well represented with a height field without occluded parts.…”

“…In particular, we want to estimate the K camera-pose matrices which represent 3D rotations to transform the point cloud in order to perform an orthogonal projection. Inspired by [21], we start from the recent method PointNet [17], which proposes a network architecture that allows for processing unordered 3D point sets, like our input. The prediction of the views should respect crucial properties such as invariance to permutations of the input data and invariance under transformations.…”

“…Secondly, another differentiable module generates the corresponding depth images of the point cloud rendered from those views. These depth images are produced by extending the work presented in [21] to handle point clouds with multiple layers of depth, occlusions and overlapping structures. Finally, a third component combines the images and uses an image classification CNN [9] to classify them.…”