Completing 3D object shape from one depth image

Rock, Jason; Gupta, Tanmay; Thorsen, Justin; Gwak, JunYoung; Shin, Daeyun; Hoiem, Derek

doi:10.1109/cvpr.2015.7298863

Cited by 163 publications

(129 citation statements)

References 37 publications

Supporting

Mentioning

126

Contrasting

Unclassified

Order By: Relevance

“…In terms of recovering a complete 3D model from a single depth map, the most similar work to ours is [23]. It introduces a learning-based algorithm to predict the unseen shape from a synthetic depth image.…”

Section: Related Workmentioning

confidence: 98%

“…We compare our algorithm with the state-of-theart methods [2], [3], [23] on a synthetic example with ground truth (Figure 10) as well as real data ( Figure 12). For fair comparisons, we make several important modifications for [2], [3], as they are originally designed to fill small holes.…”

Section: Comparisonsmentioning

confidence: 99%

“…To preserve symmetry during deformation, we add a symmetry constraint term into the deformation energy as in Rock et al [23]. Specifically, for each estimated symmetric relation, we uniformly sample symmetric point pairs on the surface mesh.…”

Section: Initial Solutionmentioning

confidence: 99%

“…Note that as in Figure 10, we modify the methods of [2] and [3] to use initialization from template and add the constraints of visibility and symmetry. The method of [23] is directly adopted to incorporate the initialization, visibility and symmetry constraints. The visibility is checked based on the segmentation mask to correct values of voxels that cause occlusions before calling the marching cube algorithm.…”

Section: Comparisonsmentioning

confidence: 99%

See 3 more Smart Citations

Shape Completion from a Single RGBD Image

Shao

et al. 2017

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

We present a novel approach for constructing a complete 3D model for an object from a single RGBD image. Given an image of an object segmented from the background, a collection of 3D models of the same category are non-rigidly aligned with the input depth, to compute a rough initial result. A volumetric-patch-based optimization algorithm is then performed to refine the initial result to generate a 3D model that not only is globally consistent with the overall shape expected from the input image but also possesses geometric details similar to those in the input image. The optimization with a set of high-level constraints, such as visibility, surface confidence and symmetry, can achieve more robust and accurate completion over state-of-the art techniques. We demonstrate the efficiency and robustness of our approach with multiple categories of objects with various geometries and details, including busts, chairs, bikes, toys, vases and tables.

show abstract

Section: Related Workmentioning

confidence: 98%

Section: Comparisonsmentioning

confidence: 99%

Section: Initial Solutionmentioning

confidence: 99%

Section: Comparisonsmentioning

confidence: 99%

See 2 more Smart Citations

Shape Completion from a Single RGBD Image

Shao

et al. 2017

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…One approach is to build a database of single viewpoint depth images corresponding to known 3D models and, given a query, extract information from the closest match to reconstruct the novel object [24]. The current state-of-the-art in this area trains CNNs to predict surface normals [32] or full 3D structure [35] from 2.5D images.…”

Section: D Object Completionmentioning

confidence: 99%

Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception

Burchfiel

Konidaris

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-We introduce Bayesian Eigenobjects (BEOs), a novel object representation that is the first technique able to perform joint classification, pose estimation, and 3D geometric completion on previously unencountered and partially observed query objects. BEOs employ Variational Bayesian Principal Component Analysis (VBPCA) directly on 3D object representations to create generative and compact probabilistic models for classes of 3D objects. Using only depth information, we significantly outperform the current state-of-the-art method for joint classification and 3D completion in both accuracy and query time. Additionally, we show that BEOs are well suited for the extremely challenging task of joint classification, completion, and pose estimation on a large dataset of household objects.

show abstract

Adaptable cascaded registration for personalized maxilla completion and cleft defect volume estimation

Zhang,

Pei,

Guo

et al. 2024

Medical Physics

View full text Add to dashboard Cite

BackgroundCone‐beam computed tomography (CBCT) images provide high‐resolution insights into the underlying craniofacial anomaly in patients with cleft lip and palate (CLP), requiring non‐negligible annotation costs to measure the cleft defect for the guidance of the clinical secondary alveolar bone graft procedures. Considering the cumbersome volumetric image acquisition, there is a lack of paired CLP CBCTs and normal CBCTs for learning‐based anatomical structure restoration models. Nowadays, the registration‐based method relieves the annotation burden, though one‐shot registration and the regular mask are limited to handling fine‐grained shape variations and harmony between restored bony tissues and the defected maxilla.PurposeThis study aimed to design and evaluate a novel method for deformable partial registration of the CLP CBCTs and normal CBCTs, enabling personalized maxilla completion and cleft defect volume prediction from CLP CBCTs.MethodsWe proposed an adaptable deep registration framework for personalized maxilla completion and cleft defect volume prediction from CLP CBCTs. The key ingredient was a cascaded partial registration to exploit the maxillary morphology prior and attribute transfer. Cascaded registration with coarse‐to‐fine registration fields handled morphological variations of cleft defects and fine‐grained maxillary restoration. We designed an adaptable cleft defect mask and volumetric Boolean operators for reliable voxel filling of the defected maxilla. A total of 36 clinically obtained CLP CBCTs were used to train and validate the proposed model, among which 22 CLP CBCTs were used to generate a training dataset with 440 synthetic CBCTs by B‐spline deformation‐based data augmentation and the remaining for testing. The proposed model was evaluated on maxilla completion and cleft defect volume prediction from clinically obtained unilateral and bilateral CLP CBCTs.ResultsExtensive experiments demonstrated the effectiveness of the adaptable cleft defect mask and the cascaded partial registration on maxilla completion and cleft defect volume prediction. The proposed method achieved state‐of‐the‐art performances with the Dice similarity coefficient of 0.90 0.02 on the restored maxilla and 0.84 0.04 on the estimated cleft defect, respectively. The average Hausdorff distance between the estimated cleft defect and the manually annotated ground truth was 0.30 0.08 mm. The relative volume error of the cleft defect was 0.08. The proposed model allowed for the prediction of cleft defect maps that were in line with the ground truth in the challenging unilateral and bilateral CLP CBCTs.ConclusionsThe results suggest that the proposed adaptable deep registration model enables patient‐specific maxilla completion and automatic annotation of cleft defects, relieving tedious voxel‐wise annotation and image acquisition burdens.

show abstract

Completing 3D object shape from one depth image

Cited by 163 publications

References 37 publications

Shape Completion from a Single RGBD Image

Shape Completion from a Single RGBD Image

Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception

Adaptable cascaded registration for personalized maxilla completion and cleft defect volume estimation

Contact Info

Product

Resources

About