DART: Dense Articulated Real-Time Tracking

Abstract

Abstract-This paper introduces DART, a general framework for tracking articulated objects composed of rigid bodies connected through a kinematic tree. DART covers a broad set of objects encountered in indoor environments, including furniture and tools, and human and robot bodies, hands and manipulators. To achieve efficient and robust tracking, DART extends the signed distance function representation to articulated objects and takes full advantage of highly parallel GPU algorithms for data association and pose optimization. We demonstrate the capabilities of DART on different types of objects that have each required dedicated tracking techniques in the past. I. INTRODUCTIONThe ability to accurately track the pose of objects in real time is of fundamental importance to many areas of robotics. Applications range from navigation to planning, manipulation and human-robot interaction, all of which have received the attention of researchers working within a state-space modelbased paradigm within both computer vision and robotics. The class of objects that can be described as collections of rigid bodies chained together through a kinematic tree is quite broad, including furniture, tools, human bodies, human hands, and robot manipulators. Tracking articulated bodies from a single viewpoint and without instrumenting the object of interest still presents a significant challenge where the single viewpoint and occlusions, including self-occlusion, limit the amount of information available for pose estimation. Noisy sensor data and approximate object models pose additional problems. Finally, the objects being tracked can be highly dynamic and have many degrees of freedom, making real-time tracking difficult.Early articulated model-based tracking techniques relied on tracking 2D features such as image edges on a CPU [8,4]. Recently introduced depth cameras along with highly parallel algorithms optimized for modern GPUs have enabled new algorithms for tracking complex 3D objects in real time. Examples include KinectFusion and related efforts for 3D mapping [23,16,34], human body pose tracking [29,35,15], articulated hand tracking [24,19,26]. These approaches were developed for specific application domains and have not been demonstrated or tested on multiple tracking applications. The application-specific nature of these approaches enables their authors to show excellent performance by taking advantage of domain-specific features and constraints, but it also prevents them from serving as general tools for tracking arbitrary articulated objects. Techniques have also been developed to

Tracking objects with point clouds from vision and touch

“…Klingensmith, et al demonstrate closed-loop servoing using articulated ICP for online pose estimation [9], and Hebert, et al utilize articulated ICP for simultaneous manipulator and manipuland tracking [5]. Following a similar path, the Dense Articulated Real-Time Tracking (DART) framework [20] performs articulated object tracking from dense depth data in real time by leveraging the signed distance function (SDF) to efficiently align an articulated object model to an incoming stream of point cloud data, while balancing a free space term. These online tracking techniques show excellent performance when fed sufficiently rich data.…”

“…We take inspiration from the DART tracking system of Schmidt, et al [20] [19] and construct a single-hypothesis tracker based on an EKF. We use the same formulation, but offer novel optimization strategy.…”

“…Thus, we know that no surface on the proposed model can lie between that point and the camera. If, for a given proposed model, we produce a simulated depth image as a collection of depths im sim = {pixel [20]. The partitioning surface, S obs , is defined by the points in the measured point cloud; free space lies in front of S obs , and out-of-sight space lies behind it.…”

Guided optimisation through classification and regression for hand pose estimation

“…When using heuristics methods can become trapped in local minima which had not been considered previously, which is why learning methods are preferred. Schmidt [23] instead performed model optimisation using a Signed Distance Function, offering a general approach for body, hand and robot tracking. Similarly to our approach, Sharp [8] proposed using Fern/Jungle based discriminative learning to provide several candidate model parameters and incorporates temporal information using a model, optimised using PSO.…”

The manifold particle filter for state estimation on high-dimensional implicit manifolds

“…Depth-based trackers, such as articulated ICP [18], [19], GMAT [20], and DART [21], can track the configuration of a robot using commercially available depth sensors. DART has been extended to incorporate contact observations using a method similar to our constraint projection [22].…”