A Versatile Learning-Based 3D Temporal Tracker: Scalable, Robust, Online

Abstract

This paper proposes a temporal tracking algorithm based on Random Forest that uses depth images to estimate and track the 3D pose of a rigid object in real-time. Compared to the state of the art aimed at the same goal, our algorithm holds important attributes such as high robustness against holes and occlusion, low computational cost of both learning and tracking stages, and low memory consumption. These are obtained (a) by a novel formulation of the learning strategy, based on a dense sampling of the camera viewpoints and learning independent trees from a single image for each camera view; as well as, (b) by an insightful occlusion handling strategy that enforces the forest to recognize the object's local and global structures. Due to these attributes, we report state-of-the-art tracking accuracy on benchmark datasets, and accomplish remarkable scalability with the number of targets, being able to simultaneously track the pose of over a hundred objects at 30 fps with an off-the-shelf CPU. In addition, the fast learning time enables us to extend our algorithm as a robust online tracker for model-free 3D objects under different viewpoints and appearance changes as demonstrated by the experiments.

A Framework for Evaluating 6-DOF Object Trackers

“…In the past, geometric methods based on ICP [4,[12][13][14] were used for temporal tracking, but they lack robustness for small objects and are generally computationally expensive. Data-driven approaches such as the ones reported in [5,6,15] can learn more robust features and the use of the Random Forest regressor [16] decreases the computing overhead significantly. Other methods show that the contours of the objects in RGB and depth data provide important cues for estimating pose [1,2,17].…”

“…While challenging at first, the dataset has now essentially been solved for the RGBD case. For example, the method of Kehl et al [1] (2017) reports an average error in translation/rotation of 0.5mm/0.26 • , which is an improvement of 0.3mm/0.1 • over the work of Tan et al (2015) [5], who have themselves reported a 0.01mm/1 • improvement to the approach designed by Krull et al (2014) [6]. The state of the art on the dataset has reached a nearperfect error of 0.1mm/0.07 • [2], which highlights the need for a new dataset with more challenging scenarios.…”

Deep Model-Based 6D Pose Refinement in RGB

“…It is worth mentioning that mixtures of the two sets of methods have been proposed as well [30,6,31,24]. Recently, methods that use only depth [34] or both modalities [21,18,10] have shown that depth can make tracking more robust by providing more clues about occlusion and scale. This work aims to explore how RGB information alone can be sufficient to perform visual tasks such as 3D tracking and 6-Degree-of-Freedom (6DoF) pose refinement by means of a Convolutional Neural Network (CNN).…”

Real-Time 3D Model Tracking in Color and Depth on a Single CPU Core

“…For very high θ, the additional iterations on the coarser scales can make a difference in up to 10% which is mainly explained by the SDF rays, capturing larger spatial distances. and Tan et al [27] to us without (A) and with cloud weighting (B).…”