Manipulation 2D and 3D Reconstruction Generative Models Digital Humans Compression 903.63 KB Robotics …and Beyond! Figure 1: Contribution of this report. Following a survey of over 250 papers, we provide a review of (Part I) techniques in neural fields such as prior learning and conditioning, representations, forward maps, architectures, and manipulation, and of (Part II) applications in visual computing including 2D image processing, 3D scene reconstruction, generative modeling, digital humans, compression, robotics, and beyond. This report is complemented by a community-driven website with search, filtering, bibliographic, and visualization features.
We propose a method to compute depth maps for every sub-aperture image in a light field in a view consistent way. Previous light field depth estimation methods typically estimate a depth map only for the central sub-aperture view, and struggle with view consistent estimation. Our method precisely defines depth edges via EPIs, then we diffuse these edges spatially within the central view. These depth estimates are then propagated to all other views in an occlusion-aware way. Finally, disoccluded regions are completed by diffusion in EPI space. Our method runs efficiently with respect to both other classical and deep learning-based approaches, and achieves competitive quantitative metrics and qualitative performance on both synthetic and real-world light fields.
Advances in machine learning have increased interest in solving visual computing problems using a class of coordinate-based neural networks that parameterize physical properties of scenes or objects across space and time. These methods, which we call neural fields, have seen widespread success in applications such as 3D shape and image synthesis, animation of human bodies, 3D reconstruction, and pose estimation. Rapid progress has led to numerous papers, but a formulation and review of the problems in visual computing has not yet emerged. We provide context, mathematical grounding, and a review of 251 papers in the literature on neural fields. In Part I, we focus on neural fields techniques by identifying common components of neural field methods, including different generalization, representation, forward map, architecture, and manipulation methods. In Part II, we focus on applications of neural fields to different problems in visual computing, and beyond (e.g., robotics, audio). Our review shows the breadth of topics already covered in visual computing, both historically and in current incarnations, and highlights the improved quality, flexibility, and capability brought by neural field methods. Finally, we present a companion website that contributes a living database that can be continually updated by the community.
CCS Concepts
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.