Colorful 3d reconstruction from a single image based on deep learning

Zhu, Yuzheng; Zhang, Yaping; Feng, Qingsong

doi:10.1145/3446132.3446157

Cited by 4 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The visual results of generated 3D model using our approach and other baselines are shown in Table 2. It is observed that most of the methods based on CNN [9,14,42,44] can learn the 3D geometry correctly. However, a rough 3D model is generated by an LSTM-based method [13].…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…The popular 3D volumetric reconstruction model named 3D-Recons [42] and OCC-Net [14] have been adapted as a baseline for performance and quality evaluation. Both types of research generate a 3D model in volumetric shapes.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…OCCNet [14] uses an occupancybased network module that refines the reconstructed model. In contrast, 3D-Recons [42] used an autoencoder-based technique for 3D model construction.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

2021

View full text Add to dashboard Cite

In recent years, learning-based approaches for 3D reconstruction have gained much popularity due to their encouraging results. However, unlike 2D images, 3D cannot be represented in its canonical form to make it computationally lean and memory-efficient. Moreover, the generation of a 3D model directly from a single 2D image is even more challenging due to the limited details available from the image for 3D reconstruction. Existing learning-based techniques still lack the desired resolution, efficiency, and smoothness of the 3D models required for many practical applications. In this paper, we propose voxel-based 3D object reconstruction (V3DOR) from a single 2D image for better accuracy, one using autoencoders (AE) and another using variational autoencoders (VAE). The encoder part of both models is used to learn suitable compressed latent representation from a single 2D image, and a decoder generates a corresponding 3D model. Our contribution is twofold. First, to the best of the authors’ knowledge, it is the first time that variational autoencoders (VAE) have been employed for the 3D reconstruction problem. Second, the proposed models extract a discriminative set of features and generate a smoother and high-resolution 3D model. To evaluate the efficacy of the proposed method, experiments have been conducted on a benchmark ShapeNet data set. The results confirm that the proposed method outperforms state-of-the-art methods.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

2021

View full text Add to dashboard Cite

show abstract

“…In recent years, many researchers turn to the field of combining differentiable renderer with neural network [35]- [37]. In the unsupervised shape reconstruction work of Rezende [38], they incorporated the OpenGL renderer into a neural network for 3D mesh reconstruction and computed the gradients of the OpenGL renderer using REINFORCE [39].…”

Section: A 3d Reconstruction Based On Deep Learningmentioning

confidence: 99%

Research on 3D Reconstruction of Furniture Based on Differentiable Renderer

Miao¹,

Jiang²,

Jiang³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Due to the self-obscuration, traditional 3D reconstruction algorithms have difficulty in recovering the 3D structure of an object from a single image. With the rapid development of convolutional neural networks, 3D reconstruction based on deep learning has attracted a wide range of attention from researchers. However, it is expensive to obtain the 3D supervised data corresponding to the objects. To solve the above problems, we combine convolutional neural networks with differentiable renderer and propose the Mesh_CA in this paper, which enables reconstruction of a single image without 3D supervision data. Specifically, an ellipsoid is first initialized for each input single view, and then the features extracted by the convolutional neural network are used to guide the deformation of the ellipsoid to obtain the generated 3D object; After that, the generated object is passed into the differentiable renderer and its corresponding contour information is output; finally, calculating the error between the predicted contour and the real one, and the final 3D object is obtained after training and testing. By training and testing on five types of furniture objects on a large-scale public dataset ShapeNet, the performance of the proposed Mesh_CA surpasses current classical methods.

show abstract

Deep Learning-Based 3-D Model for the Cultural Heritage Sites in the State of Gujarat, India

Pandi,

Aggarwal

2023

Algorithms for Intelligent Systems

View full text Add to dashboard Cite

Colorful 3d reconstruction from a single image based on deep learning

Cited by 4 publications

References 18 publications

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

Research on 3D Reconstruction of Furniture Based on Differentiable Renderer

Deep Learning-Based 3-D Model for the Cultural Heritage Sites in the State of Gujarat, India

Contact Info

Product

Resources

About