NeuralODF: Learning Omnidirectional Distance Fields for 3D Shape Representation

Abstract

In visual computing, 3D geometry is represented in many different forms including meshes, point clouds, voxel grids, level sets, and depth images. Each representation is suited for different tasks thus making the transformation of one representation into another (forward map) an important and common problem. We propose Omnidirectional Distance Fields (ODFs), a new 3D shape representation that encodes geometry by storing the depth to the object's surface from any 3D position in any viewing direction. Since rays are the fundamental unit of an ODF, it can be used to easily transform to and from common 3D representations like meshes or point clouds. Different from level set methods that are limited to representing closed surfaces, ODFs are unsigned and can thus model open surfaces (e.g., garments). We demonstrate that ODFs can be effectively learned with a neural network (Neu-ralODF) despite the inherent discontinuities at occlusion boundaries. We also introduce efficient forward mapping algorithms for transforming ODFs to and from common 3D representations. Specifically, we introduce an efficient Jumping Cubes algorithm for generating meshes from ODFs. Experiments demonstrate that NeuralODF can learn to capture high-quality shape by overfitting to a single object, and also learn to generalize on common shape categories.

Efficient Visibility Approximation for Game AI using Neural Omnidirectional Distance Fields

“…If we consider moving the origin position 𝑝 𝑟 of a ray along it's direction d𝑟 , the new ray 𝑟 ′ = (𝑝 𝑟 ′ , d𝑟 ) is an aliased version of 𝑟 . This aliasing effect can also be observed from the ODF perspective, as highlighted by [Houchens et al 2022] with the equation: we can rewrite it as…”

“…Our approach models the omnidirectional distance originating from these positions, allowing us to efficiently test occluders between them and arbitrary target positions. In particular, previous work has introduced the concept of neural ODF for 3D reconstruction using recursive inference [Houchens et al 2022], the computation time of which is impractical for our task.…”

Cycle-Consistent Generative Rendering for 2D-3D Modality Translation

“…Finally, several works eschew volume rendering itself. A number of representations [60], [61], [62], [63], [64], [65] use only a single sample per pixel, but struggle with geometric consistency and scalability. Similarly, one can move to a mesh-based representation and use rasterization instead [22], [66], [31]; however, this loses certain properties, such as amenability to further optimization or differentiable neural editing.…”