Visual end-effector tracking using a 3D model-aided particle filter for humanoid robot platforms

Abstract

Abstract-This paper addresses recursive markerless estimation of a robot's end-effector using visual observations from its cameras. The problem is formulated into the Bayesian framework and addressed using Sequential Monte Carlo (SMC) filtering. We use a 3D rendering engine and Computer Aided Design (CAD) schematics of the robot to virtually create images from the robot's camera viewpoints. These images are then used to extract information and estimate the pose of the end-effector. To this aim, we developed a particle filter for estimating the position and orientation of the robot's end-effector using the Histogram of Oriented Gradient (HOG) descriptors to capture robust characteristic features of shapes in both cameras and rendered images. We implemented the algorithm on the iCub humanoid robot and employed it in a closed-loop reaching scenario. We demonstrate that the tracking is robust to clutter, allows compensating for errors in the robot kinematics and servoing the arm in closed loop using vision.

Transferring Visuomotor Learning from Simulation to the Real World for Robotics Manipulation Tasks

“…To find the end-effector pose using visual information, it has been suggested to compare the hand perceived by the robot's camera with a realistically rendered hand from simulation [12], [13]. A particle filter is then used to predict the 6D pose of the robot's hand, which is used for a visual servoing reaching task.…”

Robot End Effector Tracking Using Predictive Multisensory Integration

“…This algorithm did not require prior knowledge of the arm model, and was robust to changes in the appearance of the end effector. Other marker-less approaches have relied upon knowledge of a 3D CAD model of the end effector (Vicente et al, 2016 ; Fantacci et al, 2017 ). Vicente et al ( 2016 ) eliminated calibration errors using a particle filter.…”

“…The likelihood associated with each particle was evaluated by comparing the outputs of Canny edge detectors applied to both the real and simulated camera images. Fantacci et al ( 2017 ) extended this particle filter and 3D CAD model based approach to estimate the end effector pose. The likelihood was evaluated using a Histogram of Oriented Gradient (HOG) (Dalal and Triggs, 2005 ) based transformation to compare the two images.…”

Real-time Pipeline for Object Modeling and Grasping Pose Selection via Superquadric Functions

“…by using the approach described in Fantacci et al (2017), which provides a precise estimate of the robot end-effector pose over time and a visual servoing approach without the use of markers. Another extension of the modeling pipeline consists in using the recognition system 10 described in Pasquale et al (2016) to classify the objects of interest according to their geometric property for using some 10 https://github.com/robotology/onthefly-recognition.…”