Segmentation-Driven 6D Object Pose Estimation

Abstract

The most recent trend in estimating the 6D pose of rigid objects has been to train deep networks to either directly regress the pose from the image or to predict the 2D locations of 3D keypoints, from which the pose can be obtained using a PnP algorithm. In both cases, the object is treated as a global entity, and a single pose estimate is computed. As a consequence, the resulting techniques can be vulnerable to large occlusions.In this paper, we introduce a segmentation-driven 6D pose estimation framework where each visible part of the objects contributes a local pose prediction in the form of 2D keypoint locations. We then use a predicted measure of confidence to combine these pose candidates into a robust set of 3D-to-2D correspondences, from which a reliable pose estimate can be obtained. We outperform the state-ofthe-art on the challenging Occluded-LINEMOD and YCB-Video datasets, which is evidence that our approach deals well with multiple poorly-textured objects occluding each other. Furthermore, it relies on a simple enough architecture to achieve real-time performance. Input CNNResult

Cutting Pose Prediction from Point Clouds

“…A similar approach that is better suited for cluttered scenes divides images into patches where the corresponding object and 2D projections are predicted for each patch. Predictions are propagated across patches and used to build a robust set of 3D-to-2D correspondences [17]. Finally, feature representations are widely used.…”

“…While the feature-based methods typically output the wanted pose directly, methods that produce intermediate representations must employ a final step to produce usable poses. For going from key points to 6-DoF poses, this is typically achieved using a Perspective-n-Point (PnP) algorithm [16,17]. A recent comprehensive survey that covered the different aspects involved in robotic grasping can be found in [23].…”

2D-Key-Points-Localization-Driven 3D Aircraft Pose Estimation

“…Recent approaches, such as R-CNN [17], Fast R-CNN [18], Faster R-CNN [19] and YOLO [20], show amazing performance on detection task. As for Key-points localization problem, it has attracted considerable study in recent years [21] [12] and precise model of target to be detected [16] [36]. As for aircraft pose estimation situation, the depth information is hard to collect due to the long distance between the sensor and the target, also the model of the object is not available if it's non-cooperative object.…”

Multi-View Shape Estimation of Transparent Containers

“…an object whose shape was not seen during training). Table 1 compares relevant works, and comprehensive reviews of object pose estimation can be found in [5,6,16,17]. Although DNN models estimate the 6 DoF object pose quite accurately, their training requires large amount of data usually annotated only for the high-level object category, containing images and/or known dense 3D models [5,6,7,16,17].…”

“…Table 1 compares relevant works, and comprehensive reviews of object pose estimation can be found in [5,6,16,17]. Although DNN models estimate the 6 DoF object pose quite accurately, their training requires large amount of data usually annotated only for the high-level object category, containing images and/or known dense 3D models [5,6,7,16,17]. For example, PoseCNN [18], DenseFusion [5], SegOPE [17] and PVNet [6] evaluate only on objects with high-quality 3D models and good visibility in depth [18].…”