Point-NeRF: Point-based Neural Radiance Fields

Xu, Qiangeng; Xu, Zexiang; Philip, Julien; Bi, Sai; Shu, Zhixin; Sunkavalli, Kalyan; Neumann, Ulrich

doi:10.48550/arxiv.2201.08845

Cited by 6 publications

(10 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a representative work, NeRF [15] pushes the neural scene representation to a new high level [27,28]. Related works based on the NeRF, including NeRF in the wild [29], NeRF body [30], and Point-NeRF [31], Human-NeRF [32], extend the NeRF to various applications. A comprehensive review of neural rendering has been reported in [33].…”

Section: Learning-based Methodsmentioning

confidence: 99%

The Cube Surface Light Field for Interactive Free-Viewpoint Rendering

Wang

2022

Applied Sciences

View full text Add to dashboard Cite

Free-viewpoint rendering has always been one of the key motivations of image-based rendering and has broad application prospects in the field of virtual reality and augmented reality (VR/AR). The existing methods mainly adopt the traditional image-based rendering or learning-based frameworks, which have limited viewpoint freedom and poor time performance. In this paper, the cube surface light field is utilized to encode the scenes implicitly, and an interactive free-viewpoint rendering method is proposed to solve the above two problems simultaneously. The core of this method is a pure light ray-based representation using the cube surface light field. Using a fast single-layer ray casting algorithm to compute the light ray’s parameters, the rendering is achieved by a GPU-based three-dimensional (3D) compressed texture mapping that converts the corresponding light rays to the desired image. Experimental results show that the proposed method can real-time render the novel views at arbitrary viewpoints outside the cube surface, and the rendering results preserve high image quality. This research provides a valid experimental basis for the potential application value of content generation in VR/AR.

show abstract

Section: Learning-based Methodsmentioning

confidence: 99%

The Cube Surface Light Field for Interactive Free-Viewpoint Rendering

Wang

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…This hybrid representation enables direct manipulation of the object geometry by editing the point cloud or the mesh extracted from our representation for fine-grained rigid or non-rigid body transformation. Unlike prior work Xu et al (2022), SPIDR estimates SDF which is critical to (i) extract higher quality meshes and surfaces for deformation and (ii) obtain better light visibility and surface normals for scene illumination.…”

Section: Geometry Editingmentioning

confidence: 99%

SPIDR: SDF-based Neural Point Fields for Illumination and Deformation

Liang¹,

Zhang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Implicit neural representations such as neural radiance fields (NeRFs) have recently emerged as a promising approach for 3D reconstruction and novel view synthesis. However, NeRF-based methods encode shape, reflectance, and illumination implicitly in their neural representations, and this makes it challenging for users to manipulate these properties in the rendered images explicitly. Existing approaches only enable limited editing of the scene and deformation of the geometry. Furthermore, no existing work enables accurate scene illumination after object deformation. In this work, we introduce SPIDR, a new hybrid neural SDF representation. SPIDR combines point cloud and neural implicit representations to enable the reconstruction of higher quality meshes and surfaces for object deformation and lighting estimation. To more accurately capture environment illumination for scene relighting, we propose a novel neural implicit model to learn environment light. To enable accurate illumination updates after deformation, we use the shadow mapping technique to efficiently approximate the light visibility updates caused by geometry editing. We demonstrate the effectiveness of SPIDR in enabling high quality geometry editing and deformation with accurate updates to the illumination of the scene. In comparison to prior work, we demonstrate significantly better rendering quality after deformation and lighting estimation.Recent work that aim to improve the editability of NeRFs (Yuan et al., 2022; Bao et al., 2022) involve extracting mesh representations from the NeRF representations. However, these approaches do not

show abstract

“…For the pure implicit method, rendering the final color of each pixel requires hundreds of network queries, making it hard to complete the computation in a short time. Some methods [Lombardi et al 2021;Xu et al 2022;Yu et al 2021a] exploit the sparse features of other representations such as mesh-based and primitive-based representations to improve efficiency. Other methods try to reduce the querying time and count.…”

Section: Related Work 21 Neural Radiance Field and Important Variantsmentioning

confidence: 99%

360Roam: Real-Time Indoor Roaming Using Geometry-Aware ${360^\circ}$ Radiance Fields

Huang¹,

Chen²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently neural rendering has attracted great attention and demonstrated impressive rendering quality. Such learned view synthesis methods exploit neural networks to implicitly represent the structure and appearance of captured objects. The neural radiance field, pioneered by NeRF [Mildenhall et al. 2020] is currently the most promising path. Instead of pursuing precise geometry reconstruction, NeRF utilizes volume rendering techniques and multi-layer perceptrons to regress density and view-dependent color per ray. By densely sampling along each ray, it can generate photorealistic results even when encountered with light reflection, transparent objects, and thin structures. To reduce the long training and inference time, the following NeRF-based methods Liu et al. 2020;Pumarola et al. 2021;] have made great efforts and modifications to increase rendering quality and speed. However, most of them focus on object-centric or small-scale scenes where placements of cameras are constrained during the

show abstract

Point-NeRF: Point-based Neural Radiance Fields

Cited by 6 publications

References 41 publications

The Cube Surface Light Field for Interactive Free-Viewpoint Rendering

The Cube Surface Light Field for Interactive Free-Viewpoint Rendering

SPIDR: SDF-based Neural Point Fields for Illumination and Deformation

360Roam: Real-Time Indoor Roaming Using Geometry-Aware ${360^\circ}$ Radiance Fields

Contact Info

Product

Resources

About