Learning to Reconstruct High-Quality 3D Shapes with Cascaded Fully Convolutional Networks

Cao, Yan‐Pei; Liu, Zheng-Ning; Kuang, Zhengfei; Kobbelt, Leif; Hu, Shi‐Min

doi:10.1007/978-3-030-01240-3_38

Cited by 25 publications

(13 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another way to improve the resolution of volumetric techniques is by using multi-staged approaches [25], [27], [34], [41], [42]. The first stage recovers a low resolution voxel grid, say 32 3 , using an encoder-decoder architecture.…”

Section: Coarse-to-fine Refinementmentioning

confidence: 99%

“…Han et al [41] extended Dai et al's approach by introducing a local 3D CNN to perform patch-level surface refinement. Cao et al [34], which recover in the first stage a volumetric grid of size 128 3 , take volumetric blocks of size 16 3 and predict whether they require further refinement. Blocks that require refinement are resampled into 512 3 and fed into another encoder-decoder for refinement, along with the initial coarse prediction to guide the refinement.…”

Section: Coarse-to-fine Refinementmentioning

confidence: 99%

“…For example, Dai et al [25] require nearest neighbor searches from a 3D database. Han et al [41] require 3D boundary detection while Cao et al [34] require assessing whether a block requires further refinement or not.…”

Section: Coarse-to-fine Refinementmentioning

confidence: 99%

See 2 more Smart Citations

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

300

150

View full text Add to dashboard Cite

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.

show abstract

Section: Coarse-to-fine Refinementmentioning

confidence: 99%

Section: Coarse-to-fine Refinementmentioning

confidence: 99%

See 1 more Smart Citation

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

300

150

View full text Add to dashboard Cite

show abstract

“…Early methods use volumetric representations such as occupancy maps [29,30], (truncated) Signed Distance Functions (SDF) [31,32,33,34], or Deep SDF (deepSDF) [35]. Their main advantage is that many of the existing deep learning architectures that have been designed for image analysis can easily be extended to 3D data by replacing the 2D pixel array with its 3D analogue and then processing the grid using 3D convolution and pooling operations.…”

Section: Related Workmentioning

confidence: 99%

Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images

Charrada,

Tabia,

Chetouani

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a novel deep reinforcement learning-based approach for 3D object reconstruction from monocular images. Prior works that use mesh representations are template based. Thus, they are limited to the reconstruction of objects that have the same topology as the template. Methods that use volumetric grids as intermediate representations are computationally expensive, which limits their application in real-time scenarios. In this paper, we propose a novel end-to-end method that reconstructs 3D objects of arbitrary topology from a monocular image. It is composed of of (1) a Vertex Generation Network (VGN), which predicts the initial 3D locations of the object's vertices from an input RGB image, (2) a differentiable triangulation layer, which learns in a non-supervised manner, using a novel reinforcement learning algorithm, the best triangulation of the object's vertices, and finally, (3) a hierarchical mesh refinement network that uses graph convolutions to refine the initial mesh. Our key contribution is the learnable triangulation process, which recovers in an unsupervised manner the topology of the input shape. Our experiments on ShapeNet and Pix3D benchmarks show that the proposed method outperforms the state-of-the-art in terms of visual quality, reconstruction accuracy, and computational time.

show abstract

“…CNN-based methods have been proposed to leverage parametric human body models, starting from single or multiple photos or RGBD images. Cao et al [78] introduced a cascaded 3D fully convolutional network to reconstruct implicit surface representations from noisy and incomplete depth maps in a two-stage process (see Fig. 5).…”

Section: Bodiesmentioning

confidence: 99%

VR content creation and exploration with deep learning: A survey

Wang

Lyu

et al. 2020

Comp. Visual Media

View full text Add to dashboard Cite

Virtual reality (VR) offers an artificial, computer generated simulation of a real life environment. It originated in the 1960s and has evolved to provide increasing immersion, interactivity, imagination, and intelligence. Because deep learning systems are able to represent and compose information at various levels in a deep hierarchical fashion, they can build very powerful models which leverage large quantities of visual media data. Intelligence of VR methods and applications has been significantly boosted by the recent developments in deep learning techniques. VR content creation and exploration relates to image and video analysis, synthesis and editing, so deep learning methods such as fully convolutional networks and general adversarial networks are widely employed, designed specifically to handle panoramic images and video and virtual 3D scenes. This article surveys recent research that uses such deep learning methods for VR content creation and exploration. It considers the problems involved, and discusses possible future directions in this active and emerging research area. Keywords virtual reality; deep learning; neural networks; 360 • image and video virtual content

show abstract

Learning to Reconstruct High-Quality 3D Shapes with Cascaded Fully Convolutional Networks

Cited by 25 publications

References 56 publications

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images

VR content creation and exploration with deep learning: A survey

Contact Info

Product

Resources

About