6D Pose Estimation for Textureless Objects on RGB Frames using Multi-View Optimization

Abstract

6D pose estimation of textureless shiny objects has become an essential problem in many robotic applications. Many pose estimators require high-quality depth data, often measured by structured light cameras. However, when objects have shiny surfaces (e.g., metal parts), these cameras fail to sense complete depths from a single viewpoint due to the specular reflection, resulting in a significant drop in the final pose accuracy. To mitigate this issue, we present a complete active vision framework for 6D object pose refinement and next-bestview prediction. Specifically, we first develop an optimizationbased pose refinement module for the structured light camera. Our system then selects the next best camera viewpoint to collect depth measurements by minimizing the predicted uncertainty of the object pose. Compared to previous approaches, we additionally predict measurement uncertainties of future viewpoints by online rendering, which significantly improves the next-best-view prediction performance. We test our approach on the challenging real-world ROBI dataset. The results demonstrate that our pose refinement method outperforms the traditional ICP-based approach when given the same input depth data, and our next-best-view strategy can achieve high object pose accuracy with significantly fewer viewpoints than the heuristic-based policies.

Multi-View Metal Parts Pose Estimation Based on a Single Camera

“…Yang proposes a two-stage method to estimate pose from multi-view, initially focusing on the translation, followed by the rotation part [12]. Other work views objects in the same scene and uses some method to join the objects in a consistent coordinate system.…”

“…Multi-view images containing more information and details yield improved pose estimation results. In light of this, recent studies have utilized multiple RGB images from various viewpoints to enhance the accuracy of object pose estimation [12][13][14]. Although fusing estimated poses from different views can enhance overall performance, this technique still presents challenges in specific scenarios, such as appearance ambiguities and potential occlusions.…”

A Method for Unseen Object Six Degrees of Freedom Pose Estimation Based on Segment Anything Model and Hybrid Distance Optimization

“…Initially, the FPFH features of each point in the two point clouds are constructed, represented as F(P) = {F(p) : p ∈ P} and F(Q) = {F(q) : q ∈ Q}. Then, correspondences between point pairs are established based on Equation (6), and these correspondences are not recalculated throughout the optimization process.…”

“…Recent years have witnessed notable advancements in machine vision systems in 6-DoF pose estimation. According to the input data type, these methods can be summarized into several typical method types: RGB-based [6][7][8][9][10][11][12][13], depth-based [14,15], RGB-D-based [16][17][18][19][20][21][22], and point cloud-based [23][24][25]. RGB-based methods primarily estimate the pose of an object by analyzing color images, benefiting from high-resolution and rich texture information.…”