Contact-consistent visual object pose estimation for contact-rich robotic manipulation tasks

Abstract

Purpose The purpose of this paper is to estimate the contact-consistent object poses during contact-rich manipulation tasks based only on visual sensors. Design/methodology/approach The method follows a four-step procedure. Initially, the raw object poses are retrieved using the available object pose estimation method and filtered using Kalman filter with nominal model; second, a group of particles are randomly generated for each pose and evaluated the corresponding object contact state using the contact simulation software. A probability guided particle averaging method is proposed to balance the accuracy and safety issues; third, the independently estimated contact states are fused in a hidden Markov model to remove the abnormal contact state observations; finally, the object poses are refined by averaging the contact state consistent particles. Findings The experiments are performed to evaluate the effectiveness of the proposed methods. The results show that the method can achieve smooth and accurate pose estimation results and the estimated contact states are consistent with ground truth. Originality/value This paper proposes a method to obtain contact-consistent poses and contact states of objects using only visual sensors. The method tries to recover the true contact state from inaccurate visual information by fusing contact simulations results and contact consistency assumptions. The method can be used to extract pose and contact information from object manipulation tasks by just observing the demonstration, which can provide a new way for the robot to learn complex manipulation tasks.

Research on two-stage grasping planning method for multifingered dexterous hand

“…The grasping program execution computer (with a central processing unit frequency of 2.8 GHz) was used to execute the grasping program for object recognition, offline data set generation of grasping solutions, grasping quality evaluation and online grasping solution selection. The Letv camera was used to capture images of objects, and the Halcon operator estimated the object’s pose based on the captured images (Tian et al , 2022). The computer that controls Sawyer was used to run the robot motion program.…”