Implicit Autoencoder for Point Cloud Self-supervised Representation Learning

Abstract

Many 3D representations (e.g., point clouds) are discrete samples of the underlying continuous 3D surface. This process inevitably introduces sampling variations on the underlying 3D shapes. In learning 3D representation, the variations should be disregarded while transferable knowledge of the underlying 3D shape should be captured. This becomes a grand challenge in existing representation learning paradigms. This paper studies autoencoding on point clouds. The standard autoencoding paradigm forces the encoder to capture such sampling variations as the decoder has to reconstruct the original point cloud that has sampling variations. We introduce Implicit Autoencoder(IAE), a simple yet effective method that addresses this challenge by replacing the point cloud decoder with an implicit decoder. The implicit decoder outputs a continuous representation that is shared among different point cloud sampling of the same model. Reconstructing under the implicit representation can prioritize that the encoder discards sampling variations, introducing more space to learn useful features. We theoretically justify this claim under a simple linear autoencoder. Moreover, the implicit decoder offers a rich space to design suitable implicit representations for different tasks. We demonstrate the usefulness of IAE across various self-supervised learning tasks for both 3D objects and 3D scenes. Experimental results show that IAE consistently outperforms the state-of-the-art in each task. Our code will be available at https://github.com/SimingYan/IAE.

3D single-cell shape analysis using geometric deep learning

“…Geometric deep learning has predominantly been used on point cloud data to produce state-of-the-art results for classification and representation learning tasks [38, 40]. Our model incorporated edge convolution [25] as the primary operator in our encoder part of our autoencoder as this has proved successful in representation learning tasks [55]. FoldingNet is a novel folding-based decoder designed to assist in representation learning on point clouds [31].…”

“…Since the creation of 3D object datasets such as ModelNet [3], the two broad approaches used to represent the input data are voxels (the 3D counterpart to pixels in 2D images) and point clouds. Recently point clouds have been the dominant approach [31,9,32]. One reason for this is practical, and another is performance.…”

“…Following similar procedures to [31, 69, 70, 71, 55], we used the trained DFN to extract features from each cell. These features were then used to train a linear support vector machine (SVM).…”

Sampled Transformer for Point Sets

“…OcCo (Wang et al, 2021) 93.0% STRL (Huang et al, 2021) 93.1% IAE (Yan et al, 2022) 93.7% [ST]Transformer-OcCo (Yu et al, 2022) 92.1% [ST]Point-BERT (Yu et al, 2022) 93.2% [ST]Point-MAE (Pang et al, 2022) 93.8%…”

DeepMend: Learning Occupancy Functions to Represent Shape for Repair

“…A large body of recent work focuses on using neural networks to represent point samples of values that implicitly define surfaces, e.g., occupancy [6,7,14,22,26,28,32,36,37,46,52,53], signed distance field (SDF) [4,13,20,27,31,35,42,48,50,51,54,58], unsigned distance field [47], or level sets [15]. These approaches show high reconstruction fidelity due to their ability to represent the continuous domain of points, while remaining computationally tractable.…”