Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance

Yariv, Lior; Kasten, Yoni; Moran, Dror; Galun, Meirav; Atzmon, Matan; Lipman, Yaron

doi:10.48550/arxiv.2003.09852

Cited by 10 publications

(38 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Novel view synthesis is a long-standing problem in computer vision and graphics, which plays a crucial role in various practical applications, including gaming, movie production, and virtual/augment reality. Recently, it has made great strides thanks to the advances in differentiable neural rendering [35,64], especially the neural radiance fields (NeRF) [29] that simplifies novel view synthesis to an optimization problem over a dense set of ground truth views.…”

Section: Introductionmentioning

confidence: 99%

“… 64. 24.47 23.95 19.91 20.86 23.27 20.78 23.44 23.35 21.53 24.18 25.09 22.70 ↑ PSNR SRN [46] 26.62 22.20 23.42 24.40 21.85 19.07 22.17 21.04 24.95 23.65 22.45 20.87 25.86 23.28 pixelNeRF [66] 29.76 26.35 27.72 27.58 23.84 24.22 28.58 24.44 30.60 26.94 25.59 27.13 29.18 26.80 Ours 31.32 27.43 28.40 28.12 24.37 24.61 28.73 24.44 30.82 27.42 26.60 26.99 29.92 27.48 Quantitative comparison on category-agnostic view synthesis.…”

mentioning

confidence: 99%

See 1 more Smart Citation

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

Yu¹,

Tang²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis. Previous solutions, such as pix-elNeRF [66], rely only on pixel-aligned features and suffer from feature ambiguity issues. As a result, they struggle with the disentanglement of geometry and appearance, leading to implausible geometries and blurry results. To address this challenge, we propose to incorporate explicit geometry reasoning and combine it with pixel-aligned features for radiance field prediction. Specifically, in addition to pixel-aligned features, we further constrain the radiance field learning to be conditioned on i) voxel-aligned features learned from a coarse volumetric grid and ii) fine surfacealigned features extracted from a regressed point cloud. We show that the introduction of such geometry-aware features helps to achieve a better disentanglement between appearance and geometry, i.e. recovering more accurate geometries and synthesizing higher quality images of novel views. Extensive experiments against state-of-the-art methods on ShapeNet benchmarks demonstrate the superiority of our approach for single-image novel view synthesis.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

Yu¹,

Tang²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The seminal papers of this category appeared in 2020 /2021. They are: NeRF [37], MNSR [54], among many others.…”

Section: Continuous Volumetric Functionsmentioning

confidence: 99%

Differential Geometry for Neural Implicit Models

Novello¹,

Silva²,

Lopes³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a neural implicit framework that bridges discrete differential geometry of triangle meshes and continuous differential geometry of neural implicit surfaces. It exploits the differentiable properties of neural networks and the discrete geometry of triangle meshes to approximate them as the zero-level sets of neural implicit functions.To train a neural implicit function, we propose a loss function that allows terms with high-order derivatives, such as the alignment between the principal directions, to learn more geometric details.During training, we consider a non-uniform sampling strategy based on the discrete curvatures of the triangle mesh to access points with more geometric details. This sampling implies faster learning while preserving geometric accuracy.We present the analytical differential geometry formulas for neural surfaces, such as normal vectors and curvatures. We use them to render the surfaces using sphere tracing. Additionally, we propose a network optimization based on singular value decomposition to reduce the number of parameters.S KdS = 2πχ(S), where K is the Gaussian curvature , and χ(S) is the Euler characteristic of S. This work proposes a sampling based on the curvature to select points with important geometric details during the training.

show abstract

“…These representations include dense point clouds [51,40,73,49,50,72,56,69,70,17,36], polygonal meshes [30,6,19,29,24,62,12,38,27,53], manifold atlases [63,15,26,18,3], and voxel grids [10,60,28,67,61,23]. While our method focuses on shape reconstruction from points, past work has used neural fields to perform a variety of 3D tasks such as shape compression [57,64], shape prediction from images [41,37], voxel grid upsampling [48,41], reconstruction from rotated inputs [14] and articulated poses [13,71], and video to 3D [68,39].…”

Section: Related Workmentioning

confidence: 99%