Articulated Multi-body Tracking under Egomotion

Abstract

In this paper, we address the problem of 3D articulated multi-person tracking in busy street scenes from a moving, human-level observer. In order to handle the complexity of multi-person interactions, we propose to pursue a twostage strategy. A multi-body detection-based tracker first analyzes the scene and recovers individual pedestrian trajectories, bridging sensor gaps and resolving temporary occlusions. A specialized articulated tracker is then applied to each recovered pedestrian trajectory in parallel to estimate the tracked person's precise body pose over time. This articulated tracker is implemented in a Gaussian Process framework and operates on global pedestrian silhouettes using a learned statistical representation of human body dynamics. We interface the two tracking levels through a guided segmentation stage, which combines traditional bottom-up cues with top-down information from a human detector and the articulated tracker's shape prediction. We show the proposed approach's viability and demonstrate its performance for articulated multi-person tracking on several challenging video sequences of a busy inner-city scenario.

Parsing human skeletons in an operating room

“…In 3D human pose estimation, there have been several approaches for multiple human pose estimation using monocular [5,31,60], stereo [21,40] or multi-view setup [27,[33][34][35]. Moreover, the problem of 3D pose estimation has been often combined with tracking [31,60].…”

Multiple Human Pose Estimation with Temporally Consistent 3D Pictorial Structures

“…In the 3D space, the problem of multiple human pose estimation has been addressed using monocular [5,18,29], stereo [14,23] or multi-camera input information [21,20]. In [29] and [18], the monocular 3D pose estimation has been combined with tracking.…”

“…Since the approach is monocular, the final 3D pose is inferred by 2D pose lifting. In [14], a two-stage algorithm is applied on stereo data for detecting human and recovering their pose. Similar to our framework, a multi-view system has been employed in [20,21].…”

“…In every application, the human motion remains consistent over time. Different approaches have been proposed for multiple human pose estimation on 2D [1,4,10,24] or 3D space [5,6,14,20,21]. Nevertheless, the utility of temporal consistency for pose estimation has not been sufficiently addressed yet.…”

Breaking the Chain: Liberation from the Temporal Markov Assumption for Tracking Human Poses

“…They are very good at tracking people in challenging videos, but they cannot cope with novel actions and generally do not adapt to style. [8] learns the correspondence between silhouette and 3D pose, as well as walk cycle. A stereo camera is used for the segmentation, and a skeleton is fit to the silhouette with the help of the dynamic priors.…”