NeRF-Editing: Geometry Editing of Neural Radiance Fields

Abstract

Implicit neural rendering, especially Neural Radiance Field (NeRF), has shown great potential in novel view synthesis of a scene. However, current NeRF-based methods cannot enable users to perform user-controlled shape deformation in the scene. While existing works have proposed some approaches to modify the radiance field according to the user's constraints, the modification is limited to color editing or object translation and rotation. In this paper, we propose a method that allows users to perform controllable shape deformation on the implicit representation of the scene, and synthesizes the novel view images of the edited scene without re-training the network. Specifically, we establish a correspondence between the extracted explicit mesh representation and the implicit neural representation of the target scene. Users can first utilize welldeveloped mesh-based deformation methods to deform the mesh representation of the scene. Our method then utilizes user edits from the mesh representation to bend the camera rays by introducing a tetrahedra mesh as a proxy, obtaining the rendering results of the edited scene. Extensive experiments demonstrate that our framework can achieve ideal editing results not only on synthetic data, but also on real scenes captured by users.

vMAP: Vectorised Object Mapping for Neural Field SLAM

“…For 2D novel view synthesis, we use the Ray-Box Intersection algorithm [10] to calculate near and far bounds for each object, and then rank rendered depths along each ray to achieve occlusion-aware scene-level rendering. This disentangled representation also opens up other types of fine-grained object-level manipulation, such as changing object shape or textures by conditioning on disentangled pre-trained feature fields [16,34], which we consider as an interesting future direction.…”

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

“…Although the pre-training process is similar to ours, the aimed task, the specific methods used, and the implementation of pretraining are different. In addition to the above extensions, NeRF has been extended for dynamic scenes [37], [38], better rendering effects [39], [40], generalization on multiple scenes [41], [42], [43], [44], [45], faster training or inference speed [46], [47], [48], [49], [50], [51], [52], re-lighting rendering [53], [54], [55], geometry or appearance editing [56], [57], [58], [59], [60] and specifically for processing human bodies [61], [62], [63] and faces [64], [65]. Some works are briefly summarized in [66].…”

Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video

“…Similar with other NeRF based face modeling approaches like Ner-FACE and NHA, artifacts may occur in some local regions when extrapolating the expression coefficients to a value that is far from the training distribution. This problem might be circumvented by explicitly modeling the underlying geometry like some NeRF Editing approaches [Bao and Yang et al 2022;Yuan et al 2022], and we leave this as a future work.…”