Single-view Object Shape Reconstruction Using Deep Shape Prior and Silhouette

Abstract

3D shape reconstruction from a single image is a highly ill-posed problem. Modern deep learning based systems try to solve this problem by learning an end-to-end mapping from image to shape via a deep network. In this paper, we aim to solve this problem via an online optimization framework inspired by traditional methods. Our framework employs a deep autoencoder to learn a set of latent codes of 3D object shapes, which are fitted by a probabilistic shape prior using Gaussian Mixture Model (GMM). At inference, the shape and pose are jointly optimized guided by both image cues and deep shape prior without relying on an initialization from any trained deep nets. Surprisingly, our method achieves comparable performance to state-of-the-art methods even without training an end-to-end network, which shows a promising step in this direction.

Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images

“…Such point-based representations are well suited for objects with intriguing parts and fine details. As such, an increasing number of papers, e.g., [9,10,11,12,13,14,15,39,16,17,18,20,19], explored their usage for deep learning-based reconstruction. To reconstruct point clouds from an input image, these methods also use an encoder, similar to volumetric representations.…”

“…In these works, 3D reconstruction is formulated as an inference problem taking advantage of the availability of collections of images annotated with their corresponding 3D models [3,4]. Existing deep learning methods for 3D reconstruction represent 3D models as volumes [5,6,7,8], point clouds [9,10,11,12,13,14,15,16,17,18,19,20], or triangulated meshes [21,22,23,24]. Volumetric representations are suitable for convolutional operations, which operate on regular grids.…”