Fostering Generalization in Single-view 3D Reconstruction by Learning a Hierarchy of Local and Global Shape Priors

Bechtold, Jan; Tatarchenko, Maxim; Fischer, Volker; Brox, Thomas

doi:10.1109/cvpr46437.2021.01562

Cited by 14 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work has made generalisation a clear goal. The goal of this study is to obtain a more expressive intermediate shape representation by locally assigning features and 3D points [ 116 ]. This is an object-centred approach.…”

Section: Object Reconstructionmentioning

confidence: 99%

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper

Vinodkumar,

Karabulut,

Avots

et al. 2024

Entropy

View full text Add to dashboard Cite

The research groups in computer vision, graphics, and machine learning have dedicated a substantial amount of attention to the areas of 3D object reconstruction, augmentation, and registration. Deep learning is the predominant method used in artificial intelligence for addressing computer vision challenges. However, deep learning on three-dimensional data presents distinct obstacles and is now in its nascent phase. There have been significant advancements in deep learning specifically for three-dimensional data, offering a range of ways to address these issues. This study offers a comprehensive examination of the latest advancements in deep learning methodologies. We examine many benchmark models for the tasks of 3D object registration, augmentation, and reconstruction. We thoroughly analyse their architectures, advantages, and constraints. In summary, this report provides a comprehensive overview of recent advancements in three-dimensional deep learning and highlights unresolved research areas that will need to be addressed in the future.

show abstract

Section: Object Reconstructionmentioning

confidence: 99%

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper

Vinodkumar,

Karabulut,

Avots

et al. 2024

Entropy

View full text Add to dashboard Cite

show abstract

“…To mitigate the memory footprint caused by high‐resolution voxels, space partitioning techniques (e.g., octree) were employed [ROUG17,HTM19,TDB17]. In order to further achieve infinite resolution, signed distance fields (SDF) [PFS*19,XWC*19,BTFB21, MON*19] were utilized, and the surface was extracted using the marching cubes algorithm [LC87]. In general, single‐view image‐based 3D reconstruction can be summarized as encoding input images into a specific feature space and then decoding them into various 3D representations (such as point clouds [YSR*20], voxels, SDF, etc.…”

Section: Related Workmentioning

confidence: 99%

Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints

Ma,

Zhao,

Teng

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Aiming at enhancing the rationality and robustness of the results of single‐view image‐based human reconstruction and acquiring richer surface details, we propose a multi‐level reconstruction framework based on implicit functions. This framework first utilizes the predicted SMPL model (Skinned Multi‐Person Linear Model) as a prior to further predict consistent 2.5D sketches (depth map and normal map), and then obtains a coarse reconstruction result through an Implicit Function fitting network (IF‐Net). Subsequently, with a pixel‐aligned feature extraction module and a fine IF‐Net, the strong constraints imposed by SMPL are relaxed to add more surface details to the reconstruction result and remove noise. Finally, to address the trade‐off between surface details and rationality under complex poses, we propose a novel fusion repair algorithm that reuses existing information. This algorithm compensates for the missing parts of the fine reconstruction results with the coarse reconstruction results, leading to a robust, rational, and richly detailed reconstruction. The final experiments prove the effectiveness of our method and demonstrate that it achieves the richest surface details while ensuring rationality. The project website can be found at https://github.com/MXKKK/2.5D‐MLIF.

show abstract

“…Some approaches make use of the grid nature within voxelized shape representations and build 2D-3D autoencoders based on convolutional neural networks (CNNs) (Xie et al 2019(Xie et al , 2020Popov, Bauszat, and Ferrari 2020). Conversely, some have focused on improving the underlying representation of 3D shapes by creating implicit functions (Mescheder et al 2019;Bechtold et al 2021), while others emphasize on the learning of alternative 3D representations such as point clouds (Fan, Su, and Guibas 2017;Lin, Kong, and Lucey 2018;Mandikal and Babu 2019) and meshes (Wang et al 2018;Wen et al 2019;Gkioxari, Malik, and Johnson 2019;Kuo et al 2020). The idea of learning from shape priors has also been explored (Wu et al 2018;Kato and Harada 2019;Cherabier et al 2018;Wu et al 2016).…”

Section: D Reconstructionmentioning

confidence: 99%

Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction

Ta-Ying

Yang

Trigoni

et al. 2022

AAAI

View full text Add to dashboard Cite

We present a pose adaptive few-shot learning procedure and a two-stage data interpolation regularization, termed Pose Adaptive Dual Mixup (PADMix), for single-image 3D reconstruction. While augmentations via interpolating feature-label pairs are effective in classification tasks, they fall short in shape predictions potentially due to inconsistencies between interpolated products of two images and volumes when rendering viewpoints are unknown. PADMix targets this issue with two sets of mixup procedures performed sequentially. We first perform an input mixup which, combined with a pose adaptive learning procedure, is helpful in learning 2D feature extraction and pose adaptive latent encoding. The stagewise training allows us to build upon the pose invariant representations to perform a follow-up latent mixup under one-to-one correspondences between features and ground-truth volumes. PADMix significantly outperforms previous literature on few-shot settings over the ShapeNet dataset and sets new benchmarks on the more challenging real-world Pix3D dataset.

show abstract

Fostering Generalization in Single-view 3D Reconstruction by Learning a Hierarchy of Local and Global Shape Priors

Cited by 14 publications

References 21 publications

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper

Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints

Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction

Contact Info

Product

Resources

About