PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

Abstract

Figure 1: Our model transforms points sampled from a simple prior to realistic point clouds through continuous normalizing flows. The videos of the transformations can be viewed on our project website: https://www.guandaoyang.com/ PointFlow/. AbstractAs 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and * Equal contribution. allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show that our model can faithfully reconstruct point clouds and learn useful representations in an unsupervised manner. The code is available at https: //github.com/stevenygd/PointFlow.

Equivariant Flow-Based Sampling for Lattice Gauge Theory

“…Recently, there has been progress in the development of flow-based generative models which can be trained to directly produce samples from a given probability distribution; early success has been demonstrated in theories of bosonic matter, spin systems, molecular systems, and for Brownian motion [24][25][26][27][28][29][30][31][32][33][34]. This progress builds on the great success of flow-based approaches for image, text, and structured object generation [35][36][37][38][39][40][41][42], as well as non-flow-based machine learning techniques applied to sampling for physics [43][44][45][46][47][48]. If flow-based algorithms can be designed and implemented at the scale of state-of-the-art calculations, they would enable efficient sampling in lattice theories that are currently hindered by CSD.…”

Learning Shape Templates With Structured Implicit Functions

“…Learned Shape Representations: To leverage shape priors, shape reconstruction methods began representing shape as a learned feature vector, with a trained decoder to a mesh [33,13,38,14,16], point cloud [10,19,43], voxel grid [9,40,6,39], or octree [34,30,29]. Most recently, representing shape as a vector with an implicit surface function decoder has become popular, with methods such as Oc-cNet [21], ImNet [8], DeepSDF [24], and DISN [42].…”

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

“…In this work, registration between different spaces is performed with the provided sketch and the internal knowledge that comes from the training procedure. Similarly, the generative model of Yang et al [69] uses a variation of an autoencoder architecture to generate 3D point clouds by modeling them as a distribution of distributions. Concretely, their method learns the distribution of shapes at the first level, and the distribution of points given a shape at the second level.…”