Local Implicit Grid Representations for 3D Scenes

Jiang, Chung-Hsiang; Sud, Avneesh; Makadia, Ameesh; Huang, Jingwei; NieBner, Matthias; Funkhouser, Thomas

doi:10.1109/cvpr42600.2020.00604

Cited by 422 publications

(273 citation statements)

References 24 publications

Supporting

Mentioning

266

Contrasting

Unclassified

Order By: Relevance

“…They showed that the learned networks can generalize between different PDE approximation methods (e.g., Finite Difference and Finite Element methods) and between approximations that correspond to different levels of discretization. Similar to our MeshfreeFlowNet framework in using a latent context grid that can be continuously decoded into an output field, Jiang et al [32] used such representations for the computer vision task of reconstructing 3D scenes, where latent context grids are decoded into continuous implicit functions that represent the surfaces of different geometries.…”

Section: Related Workmentioning

confidence: 99%

MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

Jiang

Esmaeilzadeh

Azizzadenesheli

et al. 2020

SC20: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

We propose MeshfreeFlowNet, a novel deep learningbased super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, Mesh-freeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Partial Differential Equation (PDE) constraints to be imposed, and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal domains owing to its fully convolutional encoder.We empirically study the performance of Mesh-freeFlowNet on the task of super-resolution of turbulent flows in the Rayleigh-Bénard convection problem. Across a diverse set of evaluation metrics, we show that Mesh-freeFlowNet significantly outperforms existing baselines. Furthermore, we provide a large scale implementation of MeshfreeFlowNet and show that it efficiently scales across large clusters, achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of less than 4 minutes.We provide an open-source implementation of our method that supports arbitrary combinations of PDE constraints 1 . * Denotes equal contribution.

show abstract

Section: Related Workmentioning

confidence: 99%

MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

Jiang

Esmaeilzadeh

Azizzadenesheli

et al. 2020

SC20: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…LGCL [40] adopts a similar idea except that the space partition is defined by a set of key points. LIG [19] train a part auto-encoder to learn an embedding of local crops of 3D shapes, and then the decoder is utilized to optimize the latent code of local crops at inference time.…”

Section: Related Workmentioning

confidence: 99%

“…The implicit functions separate each point in the space into inside and outside and can generate a continuous iso-surface for each shape. Some previous works [6,25,33] apply neural network as the implicit function to generalize all shapes, while some other works [13,19] observe that the global 3D implicit representation may be hard to model and generalize because the possible global shapes are too diverse and might be too complex. So these works attempt to use the deep network to infer the local 3D representation because the local appearances of different objects are similar and easy to generalize.…”

Section: Introductionmentioning

confidence: 99%

TaylorImNet for Fast 3D Shape Reconstruction Based on Implicit Surface Function

Xiao¹,

Xu²,

Gao³

2022

Preprint

View full text Add to dashboard Cite

Benefiting from the contiguous representation ability, deep implicit functions can extract the iso-surface of a shape at arbitrary resolution. However, utilizing the neural network with a large number of parameters as the implicit function prevents the generation speed of high-resolution topology because it needs to forward a large number of query points into the network. In this work, we propose TaylorImNet inspired by the Taylor series for implicit 3D shape representation. TaylorImNet exploits a set of discrete expansion points and corresponding Taylor series to model a contiguous implicit shape field. After the expansion points and corresponding coefficients are obtained, our model only needs to calculate the Taylor series to evaluate each point and the number of expansion points is independent of the generating resolution. Based on this representation, our TaylorImNet can achieve a significantly faster generation speed than other baselines. We evaluate our approach on reconstruction tasks from various types of input, and the experimental results demonstrate that our approach can get slightly better performance than existing state-of-the-art baselines while improving the inference speed with a large margin. Our code will be released in https://github.com/svip-lab/TaylorImNet.

show abstract

“…Continuous Local Point Features. Recent work shows the advantage of using a local latent shape code over a global code: both for neural implicit surfaces [9,19,27,46] and point clouds [36]. Decoding shape from local codes significantly improves geometry quality.…”

Section: Representing Humans With Point Cloudsmentioning

confidence: 99%

The Power of Points for Modeling Humans in Clothing

Yang

Tang²,

Black

2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Unseen scan Fitted POP (point cloud)Animations with pose-dependent clothing deformation Figure 1: The Power of Points (POP) model for clothed humans. Based on a novel articulated dense point cloud representation, our cross-outfit model, named POP, produces pose-dependent shapes of clothed humans with coherent global shape and expressive local garment details. The trained model can be fitted to a single scan of an unseen subject wearing an unseen outfit, and can animate it with realistic pose-dependent clothing deformations. The results are color-coded with predicted point normals and rendered with a simple surfel-based renderer.

show abstract

Local Implicit Grid Representations for 3D Scenes

Cited by 422 publications

References 24 publications

MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

TaylorImNet for Fast 3D Shape Reconstruction Based on Implicit Surface Function

The Power of Points for Modeling Humans in Clothing

Contact Info

Product

Resources

About