SRT3D: A Sparse Region-Based 3D Object Tracking Approach for the Real World

Abstract

Region-based methods have become increasingly popular for model-based, monocular 3D tracking of texture-less objects in cluttered scenes. However, while they achieve state-of-the-art results, most methods are computationally expensive, requiring significant resources to run in real-time. In the following, we build on our previous work and develop SRT3D, a sparse region-based approach to 3D object tracking that bridges this gap in efficiency. Our method considers image information sparsely along so-called correspondence lines that model the probability of the object’s contour location. We thereby improve on the current state of the art and introduce smoothed step functions that consider a defined global and local uncertainty. For the resulting probabilistic formulation, a thorough analysis is provided. Finally, we use a pre-rendered sparse viewpoint model to create a joint posterior probability for the object pose. The function is maximized using second-order Newton optimization with Tikhonov regularization. During the pose estimation, we differentiate between global and local optimization, using a novel approximation for the first-order derivative employed in the Newton method. In multiple experiments, we demonstrate that the resulting algorithm improves the current state of the art both in terms of runtime and quality, performing particularly well for noisy and cluttered images encountered in the real world.

Iterative Corresponding Geometry: Fusing Region and Depth for Highly Efficient 3D Tracking of Textureless Objects

“…In addition, it was suggested to localize the probabilistic segmentation model [23,63,76]. Different optimization techniques such as particle filters [74], Levenberg Marquardt [44], Gauss Newton [63], or Newton with Tikhonov regularization [58,59] were also proposed. Finally, starting from the ideas of [28], the efficiency problem of region-based methods was addressed with the development of the sparse tracker SRT3D [58,59].…”

“…To ensure efficiency and avoid the rendering of the 3D model during tracking, we represent the geometry using a sparse viewpoint model [59]. In the generation process, the object is rendered from a large number of virtual cameras that are placed on the vertices of a geodesic grid all around r r s…”

“…In the following, we adopt the region-based approach of SRT3D [58,59] and modify it to incorporate a user-defined uncertainty. In general, SRT3D considers region information sparsely along so-called correspondence lines l l l, which cross the estimated object contour.…”

“…While most of those methods require significant computational resources and a detailed 3D model, they achieve promising results. For the tracking of textureless objects in cluttered environments, region-based techniques have also become very popular [45,59,63,76]. Furthermore, the emergence of consumer depth sensors has enabled additional trackers that do not rely on texture [11,27,40,54,69].…”

“…In the past, it was shown that a combination of region and depth has great potential for the tracking of textureless objects [28,47]. However, while region-based tech-niques improved greatly with respect to efficiency and quality [58,59], no recent combined approach exists. In the following, we thus build on current developments and propose ICG, a highly efficient method that fuses geometric information from region-based correspondence lines and depthbased correspondence points.…”