Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Voting

Abstract

We propose a novel keypoint voting scheme based on intersecting spheres, that is more accurate than existing schemes and allows for a smaller set of more disperse keypoints. The scheme forms the basis of the proposed RCV-Pose method for 6 DoF pose estimation of 3D objects in RGB-D data, which is particularly effective at handling occlusions. A CNN is trained to estimate the distance between the 3D point corresponding to the depth mode of each RGB pixel, and a set of 3 disperse keypoints defined in the object frame. At inference, a sphere of radius equal to this estimated distance is generated, centered at each 3D point. The surface of these spheres votes to increment a 3D accumulator space, the peaks of which indicate keypoint locations. The proposed radial voting scheme is more accurate than previous vector or offset schemes, and robust to disperse keypoints. Experiments demonstrate RCVPose to be highly accurate and competitive, achieving state-of-the-art results on LINEMOD (99.7%), YCB-Video (97.2%) datasets, and notably scoring +7.9% higher than previous methods on the challenging Occlusion LINEMOD (71.1%) dataset.

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

“…In this way, the selected keypoints could be more evenly distributed on object surface and be more texture distinguishable. More recently, Wu et al [80] argue that previous vector or offset schemes are sensitive to disperse keypoints and therefore propose model RCV-Pose. In RCV-Pose, the radial voting scheme is introduced, where for each object point, a deep learning model is used to learn several spheres.…”

ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation

“…Instance-level 6D object pose estimation only estimates the 6D pose of a particular object can be divided into five parts: direct-methods [17,47,53], keypoint-based methods [29,34,37], dense coordinate-based methods [22,27,51], refinement based methods [21,25,48] and self-supervised methods [33,42]. There are also many methods propose to utilize RGBD data as input for instance-level object pose estimation [12,13,31,46]. For more details, we refer readers to Fan et al [8] for a comprehensive overview.…”

Keypoint Cascade Voting for Point Cloud Based 6DoF Pose Estimation

“…6DoF PE aims to determine the rigid transformation (comprising the 3DoF translation and 3DoF rotation) of an object of known geometry and/or appearance within a captured scene. This problem has been intensively investigated by the research community, initially using classical analytical approaches [1,48,36], and more recently exploiting the advent of machine learning (ML) methods [21,67,66,13].…”

“…A number of recent leading ML approaches [42,20,66] have been proposed based on keypoint voting, in which the 3D scene coordinates of specific keypoints defined within an object's reference frame, are voted on and accumulated independently for each image pixel. The accuracy with which Convolutional Neural Networks (CNNs) are able to regress geometric information about the locations of key-points within a scene is a main reason for the effectiveness of these approaches, and a number of variations have emerged which take both pure RGB [41,55,58,68,69,28,67,62], as well as RGB-D [66,50,18,20] data as input.…”

“…To address this, we have developed a novel keypoint voting architecture based on RCVPose [66] called RCV-Pose3D, which partitions the segmentation and regression tasks. In previous work, segmentation and regression were trained and executed in parallel, whereas in the proposed architecture, which we call cascaded keypoint voting, these tasks are trained independently and separately, and are executed end-to-end at inference.…”