Adversarial autoencoders for compact representations of 3D point clouds

Zamorski, Maciej; Zięba, Maciej; Klukowski, Piotr; Nowak, Rafał; Kurach, Karol; Stokowiec, Wojciech; Trzciński, Tomasz Piotr

doi:10.1016/j.cviu.2020.102921

Cited by 74 publications

(60 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many point cloud synthesis works convert a point distribution to a N × 3 matrix by sampling N (N is pre-defined) points from the distribution so that existing generative models are readily applicable. For example, Gadelha et al [13] apply variational auto-encoders (VAEs) [27] and Zamorski et al [56] apply adversarial auto-encoders (AAEs) [34] to point cloud generation. Achlioptas et al [1] explore generative adversarial networks (GANs) [15,2,19] for point clouds in both raw data space and latent space of a pretrained auto-encoder.…”

Section: Related Workmentioning

confidence: 99%

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

Yang

Huang

Hao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

502

463

View full text Add to dashboard Cite

Figure 1: Our model transforms points sampled from a simple prior to realistic point clouds through continuous normalizing flows. The videos of the transformations can be viewed on our project website: https://www.guandaoyang.com/ PointFlow/. AbstractAs 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and * Equal contribution. allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show that our model can faithfully reconstruct point clouds and learn useful representations in an unsupervised manner. The code is available at https: //github.com/stevenygd/PointFlow.

show abstract

Section: Related Workmentioning

confidence: 99%

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

Yang

Huang

Hao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

502

463

View full text Add to dashboard Cite

show abstract

“…Researchers are beginning to employ AEs to represent them [52,54,55]. There are seven main models as shown in Figure 1, including FoldingNet, Point Pair Feature Network (PPFNet), PPF-FoldNet, NeuralSampler [55], GeoNet [71], 3D Adversarial Autoencoder (3dAAE) [72], and 3D Point-Capsule Networks.…”

Section: Feature Learning On Point Cloudmentioning

confidence: 99%

“…3dAAE [72] obtains the representations of 3D shapes. It has the ability of end-to-end learning the representation of 3D point clouds.…”

Section: Feature Learning On Point Cloudmentioning

confidence: 99%

Deep Learning on Point Clouds and Its Application: A Survey

Liu

Sun

et al. 2019

Sensors

180

View full text Add to dashboard Cite

Point cloud is a widely used 3D data form, which can be produced by depth sensors, such as Light Detection and Ranging (LIDAR) and RGB-D cameras. Being unordered and irregular, many researchers focused on the feature engineering of the point cloud. Being able to learn complex hierarchical structures, deep learning has achieved great success with images from cameras. Recently, many researchers have adapted it into the applications of the point cloud. In this paper, the recent existing point cloud feature learning methods are classified as point-based and tree-based. The former directly takes the raw point cloud as the input for deep learning. The latter first employs a k-dimensional tree (Kd-tree) structure to represent the point cloud with a regular representation and then feeds these representations into deep learning models. Their advantages and disadvantages are analyzed. The applications related to point cloud feature learning, including 3D object classification, semantic segmentation, and 3D object detection, are introduced, and the datasets and evaluation metrics are also collected. Finally, the future research trend is predicted.

show abstract

“…Adversarial generative networks are a fabulous way to reconstruct point clouds and demonstrate extraordinary capabilities on the ModelNet. The adversarial autoencoder models AE [57] and 3dAAE [58] represent the latent space of the point cloud and show impressive results in feature interpolation and latent space editing. Although conducted on a generated dataset, they provide the immense potential for the latent space representation of large-scale point clouds, which will be the direction of subsequent research.…”

Section: Point Cloud Reconstructionmentioning

confidence: 99%

LPD-AE: Latent Space Representation of Large-Scale 3D Point Cloud

Suo

Liu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The effective latent space representation of point cloud provides a foremost and fundamental manner that can be used for challenging tasks, including point cloud based place recognition and reconstruction, especially in large-scale dynamic environments. In this paper, we present a novel deep neural network, LPD-AE(Large-scale Place Description AutoEncoder Network), to obtain meaningful local and contextual features for the generation of latent space from 3D point cloud directly. The encoder network constructs the discriminative global descriptors to realize high accuracy and robust place recognition, which contributed by extracting the local neighbor geometric features and aggregating neighborhood relationships both in feature space and physical space. The decoder network performs hierarchical reconstruction on coarse key points and ultimately produce dense point clouds, which shows that it is capable of reconstructing a full point cloud frame from a single compact but high dimensional descriptor. Our proposed network demonstrates performance that is comparable to the state-of-the-art approaches. With the benefit of the LPD-AE, many computationally complex tasks that rely directly on point clouds can be effortlessly conducted on latent space with lower memory costs, such as relocalization, loop closure detection, and map compression reconstruction. Comprehensive validations on Oxford RobotCar dataset, KITTI dataset, and our freshly collected dataset, which contains multiple trials of repeated routes in different weather and at different times, manifest its potency for real robotic and self-driving implementation.

show abstract

Adversarial autoencoders for compact representations of 3D point clouds

Cited by 74 publications

References 30 publications

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

Deep Learning on Point Clouds and Its Application: A Survey

LPD-AE: Latent Space Representation of Large-Scale 3D Point Cloud

Contact Info

Product

Resources

About