Combining Generative and Discriminative Models in a Framework for Articulated Pose Estimation

Abstract

We develop a method for the estimation of articulated pose, such as that of the human body or the human hand, from a single (monocular) image. Pose estimation is formulated as a statistical inference problem, where the goal is to find a posterior probability distribution over poses as well as a maximum a posteriori (MAP) estimate. The method combines two modeling approaches, one discriminative and the other generative. The discriminative model consists of a set of mapping functions that are constructed automatically from a labeled training set of body poses and their respective image features. The discriminative formulation allows for modeling ambiguous, one-to-many mappings (through the use of multi-modal distributions) that may yield multiple valid articulated pose hypotheses from a single image. The generative model is defined in terms of a computer graphics rendering of poses. While the generative model offers an accurate way to relate observed (image features) and hidden (body pose) random variables, it is difficult to use it directly in pose estimation, since inference is computationally intractable. In contrast, inference with the discriminative model is tractable, but considerably less accurate for the problem of interest. A combined discriminative/generative formulation is derived that leverages the complimentary strengths of both models in a principled framework for articulated pose inference. Two efficient MAP pose estimation algorithms are derived from this formulation; the first is deterministic and the second non-deterministic. Performance of the framework is quantitatively evaluated in estimating articulated pose of both the human hand and human body.Keywords: human body pose, hand pose, nonrigid and articulated pose estimation, statistical inference, generative and discriminative models, mixture models, expectation maximization algorithm

VNect

“…using [Bogo et al 2016;Rhodin et al 2016b;Sminchisescu and Triggs 2001]. Robustness could be increased with a combination of generative and discriminative estimation [Elhayek et al 2016], even from a single input view [Rosales and Sclaroff 2006;Sminchisescu et al 2006], and egocentric perspective [Rhodin et al 2016a]. We utilize generative tracking components to ensure temporal stability, but avoid model projection through a full image formation model to speed up estimation.…”

“…Monocular RGB: Monocular generative motion capture has only been shown for short clips and when paired with strong motion priors [Urtasun et al 2006] or in combination with discriminative re-initialization [Rosales and Sclaroff 2006;Sminchisescu et al 2006], since generative reconstruction is fundamentally underconstrained. Using photo-realistic template models for model fitting enables more robust monocular tracking of simple motions, but requires more expensive offline computation [de La Gorce et al 2008].…”

A recognition-based motion capture baseline on the HumanEva II test data

“…The subset of papers attempting to recover full-body pose in three dimensions from monocular input cleaves roughly according to their use of either generative or discriminative approaches, although some recent work has attempted to combine the two in order to capitalize on the distinct advantages of each [30,37].…”

“…Constructing such a mapping requires training data of some sort; these may consist of paired images and poses, or perhaps are synthetically generated from motion-captured pose data alone. Some discriminative approaches learn a regression from appearance to pose [1,7,2,30], possibly neglecting the fact that dissimilar poses can have similar featural representations in most systems.…”

A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera

“…Hybrid strategies that combine generative and discriminative methods have proven to be a suitable methodology for pose estimation and tracking procedures, see e.g. [1,6,10,23,25,28]. In these works, the main idea is to stabilize generative optimization algorithms by a discriminative component which is implemented as a database lookup or a classification scheme.…”