NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Mildenhall, Ben; Srinivasan, Pratul P.; Tancik, Matthew; Barron, Jonathan T.; Ramamoorthi, Ravi; Ng, Ren

doi:10.48550/arxiv.2003.08934

Cited by 110 publications

(344 citation statements)

References 42 publications

Supporting

Mentioning

340

Contrasting

Order By: Relevance

“…However, for reasons discussed in [34], standard NNs fail to efficiently capture high frequencies. This has been reported, for example, in capturing volumetric density [35], occupancy [36], signed distances [37]. Analysis shows that the eigenvalue spectrum of these networks decay rapidly as a function of frequency [38].…”

Section: Proposed Methodsmentioning

confidence: 64%

Length Scale Control in Topology Optimization using Fourier Enhanced Neural Networks

Chandrasekhar¹,

Suresh²

2021

Preprint

View full text Add to dashboard Cite

Length scale control is imposed in topology optimization (TO) to make designs amenable to manufacturing and other functional requirements. Broadly, there are two types of lengthscale control in TO: exact and approximate. While the former is desirable, its implementation can be difficult, and is computationally expensive. Approximate length scale control is therefore preferred, and is often sufficient for early stages of design. In this paper we propose an approximate length scale control strategy for TO, by extending a recently proposed density-based TO formulation using neural networks (TOuNN). Specifically, we enhance TOuNN with a Fourier space projection, to control the minimum and/or maximum length scales. The proposed method does not involve additional constraints, and the sensitivity computations are automated by expressing the computations in an end-end differentiable fashion using the neural net's library. The proposed method is illustrated through several numerical experiments for single and multi-material designs. Literature ReviewPopular TO methods today include density based methods ([2, 5]), level-set methods [6], and topological sensitivity methods ([7, 8, 9]). For a comprehensive review, see [10,11]. Among these, density based methods,

show abstract

Section: Proposed Methodsmentioning

confidence: 64%

Length Scale Control in Topology Optimization using Fourier Enhanced Neural Networks

Chandrasekhar¹,

Suresh²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We approach the problem with using implicit functions [7,35,38], which have shown promise in handling problems of scale and varying topology. These implicit functions have also been used in view synthesis [34,36,56,57], which differs from our work in goals. In reconstruction, implicit functions have shown impressive results on two styles of task: fitting to a single model/scene (e.g., SIREN [45]) and predicting new single objects (e.g., PiFU [40,55]).…”

Section: Related Workmentioning

confidence: 99%

What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction

Kulkarni¹,

Johnson²,

Fouhey³

2021

Preprint

View full text Add to dashboard Cite

We present an approach for scene-level 3D reconstruction, including occluded regions, from an unseen RGB image. Our approach is trained on real 3D scans and images. This problem has proved difficult for multiple reasons; Real scans are not watertight, precluding many methods; distances in scenes require reasoning across objects (making it even harder); and, as we show, uncertainty about surface locations motivates networks to produce outputs that lack basic distance function properties. We propose a new distance-like function that can be computed on unstructured scans and has good behavior under uncertainty about surface location. Computing this function over rays reduces the complexity further. We train a deep network to predict this function and show it outperforms other methods on Matterport3D, 3D Front, and ScanNet.https://nileshkulkarni.github.io/scene_drdf

show abstract

“…The recent data-driven neural rendering techniques [1], [2], [3], [4], [5] bring huge potential for realistic human portrait modeling and rendering in novel views using only RGB images as input. Specifically, the recent approaches [5], [6] utilize neural radiance fields with volume rendering to achieve photo-realistic free-viewpoint results of complicated scenes.…”

Section: Introductionmentioning

confidence: 99%

MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging

Wang

Zhao

et al. 2021

Preprint

View full text Add to dashboard Cite

Photo-realistic neural reconstruction and rendering of the human portrait are critical for numerous VR/AR applications. Still, existing solutions inherently rely on multi-view capture settings, and the one-shot solution to get rid of the tedious multi-view synchronization and calibration remains extremely challenging. In this paper, we propose MirrorNeRF -a one-shot neural portrait free-viewpoint rendering approach using a catadioptric imaging system with multiple sphere mirrors and a single high-resolution digital camera, which is the first to combine neural radiance field with catadioptric imaging so as to enable one-shot photo-realistic human portrait reconstruction and rendering, in a low-cost and casual capture setting. More specifically, we propose a light-weight catadioptric system design with a sphere mirror array to enable diverse ray sampling in the continuous 3D space as well as an effective online calibration for the camera and the mirror array. Our catadioptric imaging system can be easily deployed with a low budget and the casual capture ability for convenient daily usages. We introduce a novel neural warping radiance field representation to learn a continuous displacement field that implicitly compensates for the misalignment due to our flexible system setting. We further propose a density regularization scheme to leverage the inherent geometry information from the catadioptric data in a self-supervision manner, which not only improves the training efficiency but also provides more effective density supervision for higher rendering quality. Extensive experiments demonstrate the effectiveness and robustness of our scheme to achieve one-shot photo-realistic and high-quality appearance free-viewpoint rendering for human portrait scenes.

show abstract

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Cited by 110 publications

References 42 publications

Length Scale Control in Topology Optimization using Fourier Enhanced Neural Networks

Length Scale Control in Topology Optimization using Fourier Enhanced Neural Networks

What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction

MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging

Contact Info

Product

Resources

About