Visual Articulated Tracking in the Presence of Occlusions

Abstract

This paper focuses on visual tracking of a robotic manipulator during manipulation. In this situation, tracking is prone to failure when visual distractions are created by the object being manipulated and the clutter in the environment. Current state-of-the-art approaches, which typically rely on model-fitting using Iterative Closest Point (ICP), fail in the presence of distracting data points and are unable to recover. Meanwhile, discriminative methods which are trained only to distinguish parts of the tracked object can also fail in these scenarios as data points from the occlusions are incorrectly classified as being from the manipulator. We instead propose to use the per-pixel data-to-model associations provided from a random forest to avoid local minima during model fitting. By training the random forest with artificial occlusions we can achieve increased robustness to occlusion and clutter present in the scene. We do this without specific knowledge about the type or location of the manipulated object. Our approach is demonstrated by using dense depth data from an RGB-D camera to track a robotic manipulator during manipulation and in presence of occlusions.

Learning-driven Coarse-to-Fine Articulated Robot Tracking

“…After initial optimisation, we then switch to dense but pixel-level accurate edge correspondence for accurate registration. Compared to our previous work [13], which used pixel-wise depth image segmentation as visual cues, the 2D keypoints proposed in this paper can be located behind occlusions (see Figure 1) while texture/colour edges provide sharper contours than the imprecise edges in typical depth images.…”

“…We evaluate our tracking approach using a Kuka LWR4 7 DOF arm with a Schunk SDH2 7 DOF end-effector, which is observed by a fixed Asus Xtion PRO LIVE RGB-D camera. For further details on the experimental setup we refer to [13]. We evaluate our tracking approach on four sequences that show grasping of different objects and occlusions.…”

“…These correspondences are based on the local appearance of the estimated state and change with each iteration of the optimisation. This results in many local minima which can be mitigated by introducing discriminative information [13].…”

“…The segmentation, keypoint localisation and joint distribution prediction is trained on approximately 125000 synthetically rendered depth images in the manner described in our previous work [13]. These synthetic images show the robot at different configurations sampled from a wide range of states where the palm is inside the camera frustum.…”

Multi-person Pose Tracking using Sequential Monte Carlo with Probabilistic Neural Pose Predictor

“…Object detection and tracking are important tasks for various robotics applications, such as autonomous vehicle control [1]- [4], visual SLAM [5], [6], and robotics manipulator control [7]. Multi-person pose estimation and tracking is a critical component in various applications, such as video surveillance and sports video analytics.…”