HyperFlow: Representing 3D Objects as Surfaces

Abstract

In this work, we present HyperFlow -a novel generative model that leverages hypernetworks to create continuous 3D object representations in a form of lightweight surfaces (meshes), directly out of point clouds. Efficient object representations are essential for many computer vision applications, including robotic manipulation and autonomous driving. However, creating those representations is often cumbersome, because it requires processing unordered sets of point clouds. Therefore, it is either computationally expensive, due to additional optimization constraints such as permutation invariance, or leads to quantization losses introduced by binning point clouds into discrete voxels. Inspired by mesh-based representations of objects used in computer graphics, we postulate a fundamentally different approach and represent 3D objects as a family of surfaces. To that end, we devise a generative model that uses a hypernetwork to return the weights of a Continuous Normalizing Flows (CNF) target network. The goal of this target network is to map points from a probability distribution into a 3D mesh. To avoid numerical instability of the CNF on compact support distributions, we propose a new Spherical Log-Normal function which models density of 3D points around object surfaces mimicking noise introduced by 3D capturing devices. As a result, we obtain continuous meshbased object representations that yield better qualitative results than competing approaches, while reducing training time by over an order of magnitude.

HyperCube: Implicit Field Representations of Voxelized 3D Models

“…Therefore recent works introduced 3D representations modeled as a continuous function [25]. In such a case the implicit occupancy [1,26,27], distance field [28,29] and surface parametrization [30,31,32,33] models use a neural network to represent a 3D object. These methods do not use discretization (e.g., fixed number of voxels, points, or vertices), but represent shapes in a continuous manner and handle complicated shape topologies.…”

“…In [31,32] the authors propose HyperCloud model that uses a hyper network to output weights of a generative network to create 3D point clouds, instead of generating a fixed size reconstruction. One neural network is trained to produce a continuous representation of an object.…”

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

“…In [18], [22] the authors propose to use a hypernetwork architecture to model distribution of shapes. Instead of producing a fixed number of points, hypernetwork generate many neural networks, a single network per object.…”

“…Therefore we use continuous representation of the surface. In the basic pipeline we use HyperCloud [22], yet our method is agnostic to a 3D point cloud generative model and can work with other methods, including PointFlow [28] or HyperFlow [18]. The main idea is to represent a 3D object as a neural network, which transfers uniform distribution on a 3D sphere into the 3D object's surface (see Part A in Fig.…”

“…Tadeja is with Jagiellonian University, Institute of Applied Computer Science, Kraków, Poland; T. Trzci ński is with Warsaw University of Technology, Faculty of Electronics and Information Technology, Warsaw, Poland. Email: przemyslaw.spurek@uj.edu.pl [1], [2], [3], [6], or (ii) 3D model generation [9], [10], [11], [12], [13], [14], [15], [16], [17], [18]. While the others, e.g.…”