Neural Star Domain as Primitive Representation

Abstract

Reconstructing 3D objects from 2D images is a fundamental task in computer vision. Accurate structured reconstruction by parsimonious and semantic primitive representation further broadens its application. When reconstructing a target shape with multiple primitives, it is preferable that one can instantly access the union of basic properties of the shape such as collective volume and surface, treating the primitives as if they are one single shape. This becomes possible by primitive representation with unified implicit and explicit representations. However, primitive representations in current approaches do not satisfy all of the above requirements at the same time. To solve this problem, we propose a novel primitive representation named neural star domain (NSD) that learns primitive shapes in the star domain. We show that NSD is a universal approximator of the star domain and is not only parsimonious and semantic but also an implicit and explicit shape representation. We demonstrate that our approach outperforms existing methods in image reconstruction tasks, semantic capabilities, and speed and quality of sampling high-resolution meshes.Preprint. Under review.

Discovering 3D Parts from Image Collections

“…Although several recent methods [48,37,3,10,33,36,41,24] leverage part-based representations for 3D object reasoning, they rely on either 3D object shapes or explicit part annotations as supervision. Moreover, the learned parts only serve as additional information and are not exploited to improve 3D reconstruction.…”