Real-time Body Tracking Using a Gaussian Process Latent Variable Model

Abstract

In this paper, we present a tracking framework for capturing articulated human motions in real-time, without the need for attaching markers onto the subject's body. This is achieved by first obtaining a low dimensional representation of the training motion data, using a nonlinear dimensionality reduction technique called back-constrained GPLVM. A prior dynamics model is then learnt from this low dimensional representation by partitioning the motion sequences into elementary movements using an unsupervised EM clustering algorithm. The temporal dependencies between these elementary movements are efficiently captured by a Variable Length Markov Model. The learnt dynamics model is used to bias the propagation of candidate pose feature vectors in the low dimensional space. By combining this with an efficient volumetric reconstruction algorithm, our framework can quickly evaluate each candidate pose against image evidence captured from multiple views. We present results that show our system can accurately track complex structured activities such as ballet dancing in real-time.

Combining spatial and temporal priors for articulated human tracking with online learning

“…But we involve a strong temporal prior that can handle a continuous pose variable. On the other hand, our algorithm inherits some ideas from [8,14] regarding how the temporal prior is developed for top-down prediction. However, we use a structured spatial prior that fuses part detection results to evaluate and correct the prediction.…”

“…Back Constrained-GPLVM (BC-GPLVM) [11] improves the continuity in the latent space by enforcing the local proximities in both the LD and HD spaces. Consequentially, BC-GPLVM produces a smooth motion trajectory in the latent space that can be used as a non-parametric dynamic model for human tracking [8]. All of above DR methods focus on the exploration and exploitation of temporal priors of human motion, and they do not involve spatial (kinematic) priors explicitly.…”

Coupled Action Recognition and Pose Estimation from Multiple Views

“…Among the simplest are those based on PCA (Baumberg and Hogg 1994;Sidenbladh et al 2000;Urtasun et al 2005). More complex priors include those generated from dimensionality reduction techniques such as Isomap (Tenenbaum et al 2000) (see Gall et al 2010b), LLE (Roweis and Saul 2000) (see Elgammal and Lee 2004;Jaeggli et al 2009;Lee and Elgammal 2010) and Laplacian Eigenmaps (Belkin and Niyogi 2002) (see Sminchisescu and Jepson 2004) or probabilistic latent variable models such as the commonly used GPLVM (Lawrence 2005) and GPDM (Wang et al 2008) (see Urtasun et al 2006;Moon and Pavlovic 2006;Hou et al 2007;Geiger et al 2009;Ukita et al 2009). More recently, Taylor et al (2010) introduced the use of Conditional Restricted Boltzmann Machines, composed of large collections of discrete latent variables.…”

Human Motion Change Detection by Hierarchical Gaussian Process Dynamical Model with Particle Filter

“…Leonid et al proposed a Gaussian process annealing particle filter based method to perform 3D target tracking by exploring color histogram features [13], while he focused on pose reconstruction rather than human trajectory tracking. A real time body particle tracking framework introduced by Hou [5] to capture human motion. However, he aimed to track complex motion of one target and used the motion data for the pose estimation.…”