Learning to Stylize Novel Views

Abstract

2 Verisk Analytics 3 Google Research https://hhsinping.github.io/3d_scene_stylization Input views Style image Stylized novel views Figure 1. 3D scene stylization. Given a set of images of a 3D scene (left) as well as a reference image of the desired style (middle), our method is able to modify the style of the 3D scene, and synthesize images of arbitrary novel views (right). The novel view synthesis results 1) contain the desired style and 2) are consistent across various novel views, e.g. the texture in the yellow boxes.

StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning

“…In Fig. 4, we qualitatively compare the stylized results of novel views generated by LSNV [19] and our method. The geometry representation of LSNV comes from the voxelized point clouds of COLMAP Structure from Motion (SfM) reconstruction [40].…”

“…Consistency Measurement. Following the measurement in [19], we measure the short and long term consistency using the warped LPIPS metric [59]. A view v is warped with the depth expectation estimated by NeRF.…”

“…A user study is conducted to compare the stylization and consistency quality of our method with other state-of-the-art methods. We stylize ten series of views of the 3D scenes in the T&T dataset, using different methods [8], [51], [19] and invite 50 participants (including 28 males, 22 females, aged from 18 to 45). First we showed the participants a style image and two stylized videos generated by our method and a random compared method.…”

UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

“…Many excellent works achieve this goal through texture generation [11,22,52] and semantic view synthesis [15,17]. Some recent work [5,9,16,18,20,36,54] can transfer artistic features from a single 2D image to a complete real 3D scene, thereby changing the style in the real scene.…”

“…Most of these methods focus on how to solve the consistency problem of stylized scenes. LSVN [18] proposed a point cloud-based method for consistent 3D scene stylization. StyleMesh [16] optimized an explicit texture for the reconstructed mesh of a scene and stylized it jointly from all available input images.…”