Advances in Neural Rendering

Abstract

Figure 1: This state-of-the-art report discusses a large variety of neural rendering methods which enable applications such as novel-view synthesis of static and dynamic scenes, generative modeling of objects, and scene relighting. See Section 4 for more details on the various methods. Images adapted from [

BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis

“…These representations are well-suited to gradient-based optimization of a rendering loss, so they can be effectively optimized to reconstruct detailed geometry seen in the input images. The most successful of these volumetric approaches is Neural Radiance Fields (NeRF) [Mildenhall et al 2020], which forms the basis for many state-of-the-art view synthesis methods (see Tewari et al [2022] for a review). NeRF represents a scene as a continuous volumetric field of matter that emits and absorbs light, and renders an image using volumetric ray-tracing.…”

MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes

“…As our goal is real-time view synthesis in large unbounded scenes, this discussion is focused on approaches that accelerate rendering or reconstruct large spaces. For a comprehensive overview of recent view synthesis approaches, please refer to Tewari et al [2022].…”

i2c-net: Using Instance-Level Neural Networks for Monocular Category-Level 6D Pose Estimation

“…Explicit shape representation [19] is a class of approaches that approximates a surface as a function of 2D coordinates and enhances the granularity of this approximation by increasing the number of edges, triangles, or vertices at the expense of additional processing time. Variations in the objects' topology within the same category can hamper performances due to the possible presence of holes and gaps inside the 3D model, thus leaving space for the so-called implicit surface representations [20]. Signed Distance Functions (SDF) are the most widespread implicit model that computes the distance to the closest surface for each considered 3D point and assigns a positive (negative) sign if the point is inside (outside) [21].…”