Point Cloud GAN

Li, Chun-Liang; Zaheer, Manzil; Zhang, Yang; Póczos, Barnabás; Salakhutdinov, Ruslan

doi:10.48550/arxiv.1810.05795

Cited by 59 publications

(84 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We utilize free-form deformation (FFD) (Sederberg & Parry, 1986) and radial basis function (RBF)-based deformation (Forti & Rozza, 2014) for non-linear transformations. Such deformations are also common in VR/AR games and point clouds from generative models (GAN) (Li et al, 2018a;Zhou et al, 2021). Specifically, we use multi quadratic (ϕ(x) = √ x 2 + r 2 ) and inverse multi quadratic splines (ϕ(x) = (x 2 + r 2 ) − 1 2 ) as the representative RBFs to cover a wide range of deformation types.…”

Section: Transformation Corruptions Patternsmentioning

confidence: 99%

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Sun¹,

Zhang²,

Kailkhura³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural networks on 3D point cloud data have been widely used in the real world, especially in safety-critical applications. However, their robustness against corruptions is less studied. In this paper, we present ModelNet40-C, the first comprehensive benchmark on 3D point cloud corruption robustness, consisting of 15 common and realistic corruptions. Our evaluation shows a significant gap between the performances on ModelNet40 and ModelNet40-C for state-of-the-art (SOTA) models. To reduce the gap, we propose a simple but effective method by combining PointCutMix-R and TENT after evaluating a wide range of augmentation and testtime adaptation strategies. We identify a number of critical insights for future studies on corruption robustness in point cloud recognition. For instance, we unveil that Transformer-based architectures with proper training recipes achieve the strongest robustness. We hope our in-depth analysis will motivate the development of robust training strategies or architecture designs in the 3D point cloud domain. Our codebase and dataset are included in https://github. com/jiachens/ModelNet40-C.

show abstract

Section: Transformation Corruptions Patternsmentioning

confidence: 99%

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Sun¹,

Zhang²,

Kailkhura³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A popular technique for generating point clouds is through auto-encoders Groueix et al 2018;Zhao et al 2019]. PC-GAN [Li et al 2018c] presented a technique for synthesizing point clouds using generative adversarial networks. PointGrow [Sun et al 2020] proposed an autoregressive framework for generating each point recurrently.…”

Section: Neural Point Cloud Generationmentioning

confidence: 99%

“…Deep learning has been used to successfully synthesize point clouds for shape generation [Achlioptas et al 2018;Li et al 2018c;Yang et al 2019;Cai et al 2020], shape completion [Yuan et al 2018;, and up-sampling/consolidation [Yu et al 2018a,b;Yifan et al 2019b;]. However, since standard loss functions (e.g., adversarial or Chamfer) do not trivially enable normal regression, these methods do not generate a globally consistent normal orientation.…”

Section: Introductionmentioning

confidence: 99%

Orienting point clouds with dipole propagation

et al. 2021

View full text Add to dashboard Cite

Establishing a consistent normal orientation for point clouds is a notoriously difficult problem in geometry processing, requiring attention to both local and global shape characteristics. The normal direction of a point is a function of the local surface neighborhood; yet, point clouds do not disclose the full underlying surface structure. Even assuming known geodesic proximity, calculating a consistent normal orientation requires the global context. In this work, we introduce a novel approach for establishing a globally consistent normal orientation for point clouds. Our solution separates the local and global components into two different sub-problems. In the local phase, we train a neural network to learn a coherent normal direction per patch ( i.e. , consistently oriented normals within a single patch). In the global phase, we propagate the orientation across all coherent patches using a dipole propagation. Our dipole propagation decides to orient each patch using the electric field defined by all previously orientated patches. This gives rise to a global propagation that is stable, as well as being robust to nearby surfaces, holes, sharp features and noise.

show abstract

“…In another work [30], the authors propose to model a shape by learning the gradient field of its log-density. [26] uses auto-regressive architecture to model the distribution of 3D points, while in [31] the authors leverage a GAN architecture for the same purpose. All the above methods work on raw 3D point clouds without any colors.…”

Section: Generating 3d Objectsmentioning

confidence: 99%

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

Kostiuk¹,

Stachura²,

Tadeja³

et al. 2021

Preprint

View full text Add to dashboard Cite

Designing a 3D game scene is a tedious task that often requires a substantial amount of work. Typically, this task involves synthesis, coloring, and placement of 3D models within the game scene. To lessen this workload, we can apply machine learning to automate some aspects of the game scene development. Earlier research has already tackled automated generation of the game scene background with machine learning. However, model auto-coloring remains an underexplored problem. The automatic coloring of a 3D model is a challenging task, especially when dealing with the digital representation of a colorful, multipart object. In such a case, we have to "understand" the object's composition and coloring scheme of each part. Existing single-stage methods have their own caveats such as the need for segmentation of the object or generating individual parts that have to be assembled together to yield the final model. We address these limitations by proposing a two-stage training approach to synthesize auto-colored 3D models. In the first stage, we obtain a 3D point cloud representing a 3D object, whilst in the second stage, we assign colors to points within such cloud. Next, by leveraging the so-called triangulation trick, we generate a 3D mesh in which the surfaces are colored based on interpolation of colored points representing vertices of a given mesh triangle. This approach allows us to generate a smooth coloring scheme. Experimental evaluation shows that our two-stage approach gives better results in terms of shape reconstruction and coloring when compared to traditional single-stage techniques.

show abstract

Point Cloud GAN

Cited by 59 publications

References 31 publications

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Orienting point clouds with dipole propagation

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

Contact Info

Product

Resources

About