Monocular Multiview Object Tracking with 3D Aspect Parts

Abstract

Abstract. In this work, we focus on the problem of tracking objects under significant viewpoint variations, which poses a big challenge to traditional object tracking methods. We propose a novel method to track an object and estimate its continuous pose and part locations under severe viewpoint change. In order to handle the change in topological appearance introduced by viewpoint transformations, we represent objects with 3D aspect parts and model the relationship between viewpoint and 3D aspect parts in a part-based particle filtering framework. Moreover, we show that instance-level online-learned part appearance can be incorporated into our model, which makes it more robust in difficult scenarios with occlusions. Experiments are conducted on a new dataset of challenging YouTube videos and a subset of the KITTI dataset [14] that include significant viewpoint variations, as well as a standard sequence for car tracking. We demonstrate that our method is able to track the 3D aspect parts and the viewpoint of objects accurately despite significant changes in viewpoint.

A Novel Representation of Parts for Accurate 3D Object Detection and Tracking in Monocular Images

“…Since our approach is based on object parts, it is also related to works such as [26,27,45,20] that mostly focus on category rather than instance detection. These works were mostly motivated by the success of the Deformable Part Model [7] developed for 2D detection, which was extended successfully to 3D, e.g.…”

“…Relying on parts for 3D object detection is not new [12,27,33,20,45]. The 1 All the figures of this work are best seen in colors.…”

“…3D object detection and tracking methods have undergone impressive improvements in recent years [5,27,13,4,30,25,39,17,1,45,38,20,35,44]. However, each of the current approaches has its own weaknesses: Many of these approaches [5,13,1,35] rely on a depth sensor, which would fail on metallic objects or outdoor scenes; methods based on feature points [25,17] expect textured objects; those based on edges [4,39] are sensitive to cluttered background; most of these methods [13,27,30,38,11,45,20] are not robust to occlusion.…”

“…However, each of the current approaches has its own weaknesses: Many of these approaches [5,13,1,35] rely on a depth sensor, which would fail on metallic objects or outdoor scenes; methods based on feature points [25,17] expect textured objects; those based on edges [4,39] are sensitive to cluttered background; most of these methods [13,27,30,38,11,45,20] are not robust to occlusion. We also want a method fast enough for interactive 3D applications.…”

“…Some previous methods used homographies [16,12,45] to represent a part pose, however this assumes that the object is piece-wise planar, and it is not easy to combine the homographies from several parts together to compute a better pose for the target object. Feature point-based methods simply use the 2D locations of the feature points, which wastes very useful information.…”