Robust monocular 3D object pose tracking for large visual range variation in robotic manipulation via scale-adaptive region-based method

Zhou, Jiexin; Wang, Zi; Bao, Yunna; Wang, Qiufu; Sun, Xiaoliang; Yu, Qifeng

doi:10.1177/17298806221076978

Cited by 3 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we propose a pose refinement method based on real image patch matching. Compared with some 3D model needed methods, [26][27][28] our method does not need the 3D object mesh models and solves the difficulty of generating real-world 3D models with high-quality textures. Meanwhile, based on our efficient template search method and fully convolutional patch M_R_Net, our method achieves real-time and high-precision.…”

Section: Discussionmentioning

confidence: 99%

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

Yang,

Jiang,

et al. 2024

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

A fundamental vision technique for industrial robots involves the six degrees of freedom pose estimation of target objects from a single image. However, the direct estimation of the six degrees of freedom object pose solely from a single image is subject to limited accuracy. Various refinement approaches have been proposed to improve the accuracy by utilizing rendered images from a 3D model. Nevertheless, balancing speed and accuracy in an industrial setting remains a challenge for these methods. In this study, we propose a novel six degrees of freedom pose refinement approach centered around matching real image patches. Unlike previous approaches, our method does not rely on a 3D model, resulting in increased speed and accuracy. In the offline phase, we construct an offline database using image patches obtained from real images. During the inference phase, our method initially identifies the image patch within the offline database that is closest to the initial pose. Subsequently, we refine the six degrees of freedom pose by matching the corresponding image patches from the offline database. Experimental results indicate that our six degrees of freedom pose refinement method achieves real-time capability with a frame rate of 71 Frames Per Second (FPS), along with high precision. When the threshold is set to 0.5% of the object diameter, the average distance of dots score on the test data surpasses 70%. Moreover, experiments involving gripping and assembling tasks on an industrial robot demonstrate the ability of our method to autonomously select appropriate grasping angles and positions in real time. It further generates suitable motion paths, ultimately ensuring production efficiency.

show abstract

Section: Discussionmentioning

confidence: 99%

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

Yang,

Jiang,

et al. 2024

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

Jiang,

Xue,

Zhao

et al. 2024

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

3D object instance segmentation plays a vital role in various applications such as autonomous driving, robotics and virtual reality. However, tabletop scenes exhibit diverse object complexities and size variations. The challenge is to enhance the accuracy of segmenting these scenes for multiple object instances. This limitation directly impacts robots’ capabilities to effectively grasp and manipulate objects. In this paper, we propose a multi-scale deep learning and clustering-based approach for object instance segmentation in tabletop scenes. Our approach incorporates a multi-scale neighborhood feature sampling (MNFS) module specifically designed to extract local features, and a clustering algorithm to eliminate noise and preserve instance integrity. Furthermore, we combine the strength of both methods through ScoreNet and non-maximal suppression. We conducted extensive experiments on TO-Scene, the first large-scale dataset of 3D tabletop scenes, and observed an average mIoU improvement of approximately 4.07% compared to existing methods. This highlights the superior performance of our proposed method. In addition, we tested our algorithm on a real-scene robotics platform and showed that it has good performance and generalization capabilities to support future applications such as robot grasping.

show abstract

Aircraft Carrier Pose Tracking Based on Adaptive Region in Visual Landing

Zhou

Wang

Zhang

et al. 2022

Drones

View full text Add to dashboard Cite

Due to its structural simplicity and its strong anti-electromagnetic ability, landing guidance based on airborne monocular vision has gained more and more attention. Monocular 6D pose tracking of the aircraft carrier is one of the key technologies in visual landing guidance. However, owing to the large range span in the process of carrier landing, the scale of the carrier target in the image variates greatly. There is still a lack of robust monocular pose tracking methods suitable for this scenario. To tackle this problem, a new aircraft carrier pose tracking algorithm based on scale-adaptive local region is proposed in this paper. Firstly, the projected contour of the carrier target is uniformly sampled to establish local circular regions. Then, the local area radius is adjusted according to the pixel scale of the projected contour to build the optimal segmentation energy function. Finally, the 6D pose tracking of the carrier target is realized by iterative optimization. Experimental results on both synthetic and real image sequences show that the proposed method achieves robust and efficient 6D pose tracking of the carrier target under the condition of large distance span, which meets the application requirements of carrier landing guidance.

show abstract

Robust monocular 3D object pose tracking for large visual range variation in robotic manipulation via scale-adaptive region-based method

Cited by 3 publications

References 30 publications

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

Aircraft Carrier Pose Tracking Based on Adaptive Region in Visual Landing

Contact Info

Product

Resources

About