SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again

Kehl, Wadim; Manhardt, Fabian; Tombari, Federico; Ilić, Slobodan; Navab, Nassir

doi:10.1109/iccv.2017.169

Cited by 886 publications

(820 citation statements)

References 33 publications

Supporting

Mentioning

816

Contrasting

Order By: Relevance

“…Methods working in the RGB channel in Table VIII are based on CNN structure. According to the numbers presented in Table VIII, RGB-based SSD-6D [37] and RGB-D-based LCHF achieve similar performance. These recall values show the promising performance of CNN architectures across random forest-based learning methods.…”

Section: A Analyses Based On Average Distancementioning

confidence: 94%

“…Unlike the previous categories of methods, i.e., classification-based and regressionbased, this category performs the classification and regression tasks within a single architecture. The methods can firstly do the classification, the outcomes of which are cured in a regression-based refinement step [105], [84], [78], [166] or vice versa [75], or can do the classification and regression in a single-shot process [87], [145], [101], [106], [100], [148], [103], [102], [30], [37], [162].…”

Section: B Regressionmentioning

confidence: 99%

“…Distortions along the object borders arising from occlusion and clutter, that is, the distortions of the color gradient and surface normal information in the test processes, mainly degrade the performance of this detector. Part-based methods [4], [28], [35], [30], [37] extract parts in the given image. Despite the fact that LCHF uses the same kinds of features as in Linemod, LCHF detects objects extracting parts, thus making the method more robust to occlusion and clutter.…”

Section: A Analyses Based On Average Distancementioning

confidence: 99%

“…Template-based methods, i.e., Linemod, match the features extracted during test to a set of templates, and hence, they cannot easily be generalized well to unseen ground truth annotations, that is, the translation and rotation parameters in object pose estimation. Methods based on random forests [4], [28], [35], [30] efficiently benefit the randomisation embedded in this learning tool, consequently providing good generalisation performance [37], [39]. Methods working in the RGB channel in Table VIII are based on CNN structure.…”

Section: A Analyses Based On Average Distancementioning

confidence: 99%

“…We first analyse the performance of the methods [36], [4], [2], [37] on the LINEMOD dataset. On the average, Kehl et al [36] outperforms other methods proving the superiority of learning deep features.…”

Section: A Analyses Based On Average Distancementioning

confidence: 99%

See 4 more Smart Citations

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

Şahin

Garcia-Hernando

Sock

et al. 2020

Image and Vision Computing

View full text Add to dashboard Cite

Object pose recovery has gained increasing attention in the computer vision field as it has become an important problem in rapidly evolving technological areas related to autonomous driving, robotics, and augmented reality. Existing review-related studies have addressed the problem at visual level in 2D, going through the methods which produce 2D bounding boxes of objects of interest in RGB images. The 2D search space is enlarged either using the geometry information available in the 3D space along with RGB (Mono/Stereo) images, or utilizing depth data from LIDAR sensors and/or RGB-D cameras. 3D bounding box detectors, producing category-level amodal 3D bounding boxes, are evaluated on gravity aligned images, while full 6D object pose estimators are mostly tested at instance-level on the images where the alignment constraint is removed. Recently, 6D object pose estimation is tackled at the level of categories. In this paper, we present the first comprehensive and most recent review of the methods on object pose recovery, from 3D bounding box detectors to full 6D pose estimators. The methods mathematically model the problem as a classification, regression, classification & regression, template matching, and point-pair feature matching task. Based on this, a mathematical-model-based categorization of the methods is established. Datasets used for evaluating the methods are investigated with respect to the challenges, and evaluation metrics are studied. Quantitative results of experiments in the literature are analysed to show which category of methods best performs across what types of challenges. The analyses are further extended comparing two methods, which are our own implementations, so that the outcomes from the public results are further solidified. Current position of the field is summarized regarding object pose recovery, and possible research directions are identified.

show abstract

Section: A Analyses Based On Average Distancementioning

confidence: 94%

Section: B Regressionmentioning

confidence: 99%

Section: A Analyses Based On Average Distancementioning

confidence: 99%

Section: A Analyses Based On Average Distancementioning

confidence: 99%

Section: A Analyses Based On Average Distancementioning

confidence: 99%

See 3 more Smart Citations

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

Şahin

Garcia-Hernando

Sock

et al. 2020

Image and Vision Computing

View full text Add to dashboard Cite

show abstract

A Knowledge Transfer Method for Unsupervised Pose Keypoint Detection Based on Domain Adaptation and CAD Models

Kong

Delong

2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Vision‐based pose estimation is a basic task in many industrial fields such as bin‐picking, autonomous assembly, and augmented reality. One of the most commonly used pose estimation methods first detects the 2D pose keypoints in the input image and then calculates the 6D pose using a pose solver. Recently, deep learning is widely used in pose keypoint detection and performs excellent accuracy and adaptability. However, its over‐reliance on sufficient and high‐quality samples and supervision is prominent, particularly in the industrial field, leading to high data cost. Based on domain adaptation and computer‐aided‐design (CAD) models, herein, a virtual‐to‐real knowledge transfer method for pose keypoint detection to reduce the data cost of deep learning is proposed. To address the disorder of knowledge flow, a viewpoint‐driven feature alignment strategy is proposed to simultaneously eliminate interdomain differences and preserve intradomain differences. The shape invariance of rigid objects is then introduced as constraints to address the large assumption space problem in the regressive domain adaptation. The multidimensional experimental results demonstrate the superiority of the method. Without real annotations, the normalized pixel error of keypoint detection is reported as 0.033, and the proportion of pixel errors lower than 0.05 is up to 92.77%.

show abstract

A deep learning framework for real‐time 3D model registration in robot‐assisted laparoscopic surgery

Padovan

Marullo

Tanzi

et al. 2022

Robotics Computer Surgery

View full text Add to dashboard Cite

Introduction The current study presents a deep learning framework to determine, in real‐time, position and rotation of a target organ from an endoscopic video. These inferred data are used to overlay the 3D model of patient's organ over its real counterpart. The resulting augmented video flow is streamed back to the surgeon as a support during laparoscopic robot‐assisted procedures. Methods This framework exploits semantic segmentation and, thereafter, two techniques, based on Convolutional Neural Networks and motion analysis, were used to infer the rotation. Results The segmentation shows optimal accuracies, with a mean IoU score greater than 80% in all tests. Different performance levels are obtained for rotation, depending on the surgical procedure. Discussion Even if the presented methodology has various degrees of precision depending on the testing scenario, this work sets the first step for the adoption of deep learning and augmented reality to generalise the automatic registration process.

show abstract

SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again

Cited by 886 publications

References 33 publications

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

A Knowledge Transfer Method for Unsupervised Pose Keypoint Detection Based on Domain Adaptation and CAD Models

A deep learning framework for real‐time 3D model registration in robot‐assisted laparoscopic surgery

Contact Info

Product

Resources

About