NeuTex: Neural Texture Mapping for Volumetric Neural Rendering

Xiang, Fanbo; Xu, Zexiang; Hašan, Miloš; Hold-Geoffroy, Yannick; Sunkavalli, Kalyan; Su, Hao

doi:10.1109/cvpr46437.2021.00704

Cited by 62 publications

(31 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The key difference between implicit neural networks and conventional fully connected networks is that the former can learn high frequency functions more effectively and, thus, can encode natural signals with higher fidelity. Owing to this unique ability, implicit neural networks have penetrated many tasks in computer vision such as texture generation [Henzler et al, 2020, Oechsle et al, 2019, Henzler et al, 2020, Xiang et al, 2021, shape representation [Chen and Zhang, 2019, Deng et al, 2020, Tiwari et al, 2021, Genova et al, 2020, Basher et al, 2021, Mu et al, 2021, Park et al, 2019, and novel view synthesis , Niemeyer et al, 2020, Saito et al, 2019, Sitzmann et al, 2019, Yu et al, 2021, Pumarola et al, 2021, Rebain et al, 2021, Park et al, 2021.…”

Section: Related Workmentioning

confidence: 99%

How You Start Matters for Generalization

Ramasinghe¹,

MacDonald²,

Farazi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Characterizing the remarkable generalization properties of over-parameterized neural networks remains an open problem. In this paper, we promote a shift of focus towards initialization rather than neural architecture or (stochastic) gradient descent to explain this implicit regularization. Through a Fourier lens, we derive a general result for the spectral bias of neural networks and show that the generalization of neural networks is heavily tied to their initialization. Further, we empirically solidify the developed theoretical insights using practical, deep networks. Finally, we make a case against the controversial flat-minima conjecture and show that Fourier analysis grants a more reliable framework for understanding the generalization of neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

How You Start Matters for Generalization

Ramasinghe¹,

MacDonald²,

Farazi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…NeRF [27] proposes to use a 5D function to represent the scene and applies volumetric rendering for novel view synthesis, achieving photo-realistic results and detailed geometry reconstruction. This powerful representation quickly receives attention and is extensively studied and applied in various fields [53,25], such as generative settings [6,38,30,5], dynamic scenes [21,32], and texture mapping [47]. In particular, we categorize recent progress by the design of the underlying functions into three classes: implicit, hybrid and explicit.…”

Section: Scene Representation With Nerfmentioning

confidence: 99%

Compressible-composable NeRF via Rank-residual Decomposition

Tang¹,

Chen²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering. However, its implicit representation causes difficulty in manipulating the models like the explicit mesh representation. Several recent advances in NeRF manipulation are usually restricted by a shared renderer network, or suffer from large model size. To circumvent the hurdle, in this paper, we present an explicit neural field representation that enables efficient and convenient manipulation of models. To achieve this goal, we learn a hybrid tensor rank decomposition of the scene without neural networks. Motivated by the low-rank approximation property of the SVD algorithm, we propose a rank-residual learning strategy to encourage the preservation of primary information in lower ranks. The model size can then be dynamically adjusted by rank truncation to control the levels of detail, achieving near-optimal compression without extra optimization. Furthermore, different models can be arbitrarily transformed and composed into one scene by concatenating along the rank dimension. The growth of storage cost can also be mitigated by compressing the unimportant objects in the composed scene. We demonstrate that our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition. Code will be made available at https://github.com/ashawkey/CCNeRF.Preprint. Under review.

show abstract

“…NeRF [28] uses a global MLP to regress the volume density and view-dependent radiance at any arbitrary point in the space, and applies volume rendering to synthesize images at novel viewpoints. Following works extend the framework for different tasks such as relighting [3][4][5]39], scene editing [46] and dynamic scene modeling [23,31,32]. Similar to NeRF, most of these works train MLP networks, specific for each scene from scratch, which can take hours and even days to optimize, heavily time-consuming.…”

Section: Related Workmentioning

confidence: 99%