RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation

Zhang, Ruida; Di, Yan; Lou, Zhiqiang; Manhardt, Fabian; Tombari, Federico; Ji, Xiangyang

doi:10.1007/978-3-031-19769-7_38

Cited by 28 publications

(18 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…And then, they formulate the pose estimation problem as a camera-and world-space correspondence learning problem which explicitly aligns the coordinates [30]. Although considerable progress has been attained with prior-based methods [3,40,35], the requirements of collecting a large amount of ground-truth 3D models of target objects for obtaining the 3D prior and supervising training the prior deformation module hinders their practical applicability. This motivates us to investigate the mechanism that makes prior-based methods effective.…”

Section: Prior-based Methodsmentioning

confidence: 99%

“…Fan et al [9] adopt a shape prior guided reconstruction network and a discriminator network to learn high-quality canonical representations. Zhang et al [40] use the shape priors as the indicator to predict pose and zero-mean residual vectors which encapsulate the spatial cues of the pose and enable geometry-guided consistency terms. Zhang et al [39] learn dense correspondences between input images and the canonical shape prior via surface embedding.…”

Section: Prior-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Liu¹,

Chen²,

Ye³

et al. 2023

Preprint

View full text Add to dashboard Cite

Category-level 6D pose estimation aims to predict the poses and sizes of unseen objects from a specific category. Thanks to prior deformation, which explicitly adapts a category-specific 3D prior (i.e., a 3D template) to a given object instance, prior-based methods attained great success and have become a major research stream. However, obtaining category-specific priors requires collecting a large amount of 3D models, which is labor-consuming and often not accessible in practice. This motivates us to investigate whether priors are necessary to make prior-based methods effective. Our empirical study shows that the 3D prior itself is not the credit to the high performance. The keypoint actually is the explicit deformation process, which aligns camera and world coordinates supervised by worldspace 3D models (also called canonical space). Inspired by these observation, we introduce a simple prior-free implicit space transformation network, namely IST-Net, to transform camera-space features to world-space counterparts and build correspondence between them in an implicit manner without relying on 3D priors. Besides, we design camera-and world-space enhancers to enrich the features with pose-sensitive information and geometrical constraints, respectively. Albeit simple, IST-Net becomes the first prior-free method that achieves state-of-the-art performance, with top inference speed on the REAL275 dataset. Our code and models will be publicly available.

show abstract

Section: Prior-based Methodsmentioning

confidence: 99%

Section: Prior-based Methodsmentioning

confidence: 99%

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Liu¹,

Chen²,

Ye³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Symmetry axis. Following FS-Net [29] some approaches [32,53,54] tailor the orientation representation to the predefined axis-symmetries. Specifically, these methods parameterize the rotation by two axes (similar to [69]), which is sufficient to construct a rotation matrix (i.e., the third vector must be orthogonal to the first two).…”

Section: Symmetry Handlingmentioning

confidence: 99%

“…Furthermore, self-supervised learning is employed by augmenting the input point sets in two different ways, which enables the use of a consistency loss when no ground truth is available. [54] generally follows FS-Net [29] in directly regressing the pose in a correspondence-free manner, but extends it with shape reconstruction and various additional losses. In particular, the pose is estimated by two parallel branches: first, by a simple regression branch (like FS-Net [29]); second, by predicting the deformation and assignment (like SPD [29]) and subsequently the residual pointwise bounding box projections (hence RBP; see [32] for details on pointwise bounding box projection).…”

Section: Asm-net Akizuki and Hashimotomentioning

confidence: 99%

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

Bruns¹

2023

Preprint

View full text Add to dashboard Cite

Recently, various methods for 6D pose and shape estimation of objects at a per-category level have been proposed. This work provides an overview of the field in terms of methods, datasets, and evaluation protocols. First, an overview of existing works and their commonalities and differences is provided. Second, we take a critical look at the predominant evaluation protocol, including metrics and datasets. Based on the findings, we propose a new set of metrics, contribute new annotations for the Redwood dataset, and evaluate state-of-the-art methods in a fair comparison. The results indicate that existing methods do not generalize well to unconstrained orientations and are actually heavily biased towards objects being upright. We provide an easy-to-use evaluation toolbox with well-defined metrics, methods, and dataset interfaces, which allows evaluation and comparison with various state-of-the-art approaches (https://github.com/roym899/pose and shape evaluation).

show abstract

“…robotics manipulation [2], [3], [4], and scene understanding [5]. The bottleneck in this task is scale-ambiguous distance perception [6], [7], [8] induced by the perspective projection onto the image plane.…”

mentioning

confidence: 99%

OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection

Di²,

Zhai³

et al. 2023

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Monocular 3D object detection has recently made a significant leap forward thanks to the use of pre-trained depth estimators for pseudo-LiDAR recovery. Yet, such two-stage methods typically suffer from overfitting and are incapable of explicitly encapsulating the geometric relation between depth and object bounding box. To overcome this limitation, we instead propose to jointly estimate dense scene depth with depth-bounding box residuals and object bounding boxes, allowing a two-stream detection of 3D objects that harnesses both geometry and context information. Thereby, the geometry stream combines visible depth and depth-bounding box residuals to recover the object bounding box via explicit occlusion-aware optimization. In addition, a bounding box based geometry projection scheme is employed in an effort to enhance distance perception. The second stream, named as the Context Stream, directly regresses 3D object location and size. This novel two-stream representation enables us to enforce cross-stream consistency terms, which aligns the outputs of both streams, and further improves the overall performance. Extensive experiments on the public benchmark demonstrate that OPA-3D outperforms state-of-the-art methods on the main Car category, whilst keeping a real-time inference speed.

show abstract

RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation

Cited by 28 publications

References 46 publications

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Prior-free Category-level Pose Estimation with Implicit Space Transformation

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection

Contact Info

Product

Resources

About