DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation

Wen, Yilin; Li, Xiangyu; Pan, Hao; Yang, Liwei; Wang, Zheng; Komura, Taku; Wang, Wenping

doi:10.1007/978-3-031-20077-9_24

Cited by 11 publications

(9 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Novel views. Instead of explicitly regressing the shape, Chen et al [21] and DISP6D [41] employ neural networks that generate RGB and optionally depth views given a latent shape representation and viewing direction. In contrast to neural field representations (see Continuous SDFs below), multi-view consistency is not enforced by these methods, that is, there is no guarantee that the resulting views are consistent with each other.…”

Section: Shape Representationmentioning

confidence: 99%

“…At inference time, the orientation can be inferred by finding the closest stored latent representation (and its associated orientation) to the encoded input. This approach automatically learns any form of ambiguity and has been adapted to categorical pose estimation by iCaps[22] and DISP6D[41]. iCaps uses a categorical reference object when training the decoder (i.e., implying category-level ambiguities), whereas DISP6D modifies the codebook at inference time, which in principle supports inference of object-level ambiguities.…”

mentioning

confidence: 99%

See 1 more Smart Citation

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

Bruns¹

2023

Preprint

View full text Add to dashboard Cite

Recently, various methods for 6D pose and shape estimation of objects at a per-category level have been proposed. This work provides an overview of the field in terms of methods, datasets, and evaluation protocols. First, an overview of existing works and their commonalities and differences is provided. Second, we take a critical look at the predominant evaluation protocol, including metrics and datasets. Based on the findings, we propose a new set of metrics, contribute new annotations for the Redwood dataset, and evaluate state-of-the-art methods in a fair comparison. The results indicate that existing methods do not generalize well to unconstrained orientations and are actually heavily biased towards objects being upright. We provide an easy-to-use evaluation toolbox with well-defined metrics, methods, and dataset interfaces, which allows evaluation and comparison with various state-of-the-art approaches (https://github.com/roym899/pose and shape evaluation).

show abstract

Section: Shape Representationmentioning

confidence: 99%

mentioning

confidence: 99%

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

Bruns¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Category-level pose estimation aims at generalizing to unknown objects of the same category [32], [16], [61], [69], [24], [106]. As compared to instance-level object pose estimation, which learn to predict poses of known objects.…”

Section: G Category-level Trainingmentioning

confidence: 99%

“…As compared to instance-level object pose estimation, which learn to predict poses of known objects. Common principles are encoding canonical or category-specific features [61], [16], [24], [106], render-and-compare [69] and fast adaptation [32].…”

Section: G Category-level Trainingmentioning

confidence: 99%

Open Challenges for Monocular Single-shot 6D Object Pose Estimation

Thalhammer¹,

Peter²,

Weibel³

et al. 2023

Preprint

View full text Add to dashboard Cite

Object pose estimation is a non-trivial task that enables robotic manipulation, bin picking, augmented reality, and scene understanding, to name a few use cases. Monocular object pose estimation gained considerable momentum with the rise of high-performing deep learning-based solutions and is particularly interesting for the community since sensors are inexpensive and inference is fast. Prior works establish the comprehensive state of the art for diverse pose estimation problems. Their broad scopes make it difficult to identify promising future directions. We narrow down the scope to the problem of singleshot monocular 6D object pose estimation, which is commonly used in robotics, and thus are able to identify such trends. By reviewing recent publications in robotics and computer vision, the state of the art is established at the union of both fields. Following that, we identify promising research directions in order to help researchers to formulate relevant research ideas and effectively advance the state of the art. Findings include that methods are sophisticated enough to overcome the domain shift and that occlusion handling is a fundamental challenge. We also highlight problems such as novel object pose estimation and challenging materials handling as central challenges to advance robotics.

show abstract

“…For example, ShAPO [13] jointly predicts object shape, pose, and size in a single-shot manner. DISP6D [34] disentangles the latent representation of shape and pose into two sub-spaces, improving the scalability and generality. Neural Radiance Fields (NeRF) [24] provides a mechanism for capturing complex 3D structures from only one or a few RGB images, which is also applicable to object pose estimation.…”

Section: Implicit Field For Pose Estimationmentioning

confidence: 99%

SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations

Wan¹,

Shi²,

Xu³

2023

Preprint

View full text Add to dashboard Cite

Most learning-based approaches to category-level 6D pose estimation are design around normalized object coordinate space (NOCS). While being successful, NOCS-based methods become inaccurate and less robust when handling objects of a category containing significant intra-category shape variations. This is because the object coordinates induced by global and rigid alignment of objects are semantically incoherent, making the coordinate regression hard to learn and generalize. We propose Semantically-aware Object Coordinate Space (SOCS) built by warping-andaligning the objects guided by a sparse set of keypoints with semantically meaningful correspondence. SOCS is semantically coherent: Any point on the surface of a object can be mapped to a semantically meaningful location in SOCS, allowing for accurate pose and size estimation under large shape variations. To learn effective coordinate regression to SOCS, we propose a novel multi-scale coordinate-based attention network. Evaluations demonstrate that our method is easy to train, well-generalizing for large intra-category shape variations and robust to inter-object occlusions.

show abstract

DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation

Cited by 11 publications

References 44 publications

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

Open Challenges for Monocular Single-shot 6D Object Pose Estimation

SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations

Contact Info

Product

Resources

About