Multi-feature max-margin hierarchical Bayesian model for action recognition

Abstract

In this paper, a multi-feature max-margin hierarchical Bayesian model (M 3 HBM) is proposed for action recognition. Different from existing methods which separate representation and classification into two steps, M 3 HBM jointly learns a high-level representation by combining a hierarchical generative model (HGM) and discriminative maxmargin classifiers in a unified Bayesian framework. Specifically, HGM is proposed to represent actions by distributions over latent spatial temporal patterns (STPs) which are learned from multiple feature modalities and shared among different classes. For recognition, we employ Gibbs classifiers to minimize the expected loss function based on the max-margin principle and use the classifiers as regularization terms of M 3 HBM to perform Bayeisan estimation for classifier parameters together with the learning of STPs. In addition, multi-task learning is applied to learn the model from multiple feature modalities for different classes. For test videos, we obtain the representations by the inference process and perform action recognition by the learned Gibbs classifiers. For the learning and inference process, we derive an efficient Gibbs sampling algorithm to solve the proposed M 3 HBM. Extensive experiments on several datasets demonstrate both the representation power and the classification capability of our approach for action recognition.

Retrieval of spatial–temporal motion topics from 3D skeleton data

“…For example, by including label information, Bregonzio et al [2] add a non-uniform topic proportion constraint on LDA to discover class-specific topics of actions in video. Similarly, a multi-feature Bayesian model [45] is proposed on the basis of LDA to jointly learn the mix-ture distributions of sparse and dense motion attributes. In the attempt of skeleton-based motion analysis, seven pairs of geometric features of limb actions are processed and weighted to generate the text-based motion description [51] which ignores the order of motion.…”

Watch-n-Patch: Unsupervised Learning of Actions and Relations

“…However, the independence assumption of different topics would lead to non smooth temporal segmentations. Recently, a multifeature max-margin hierarchical Bayesian model [60] is proposed to jointly learn a high-level representation by combining a hierarchical generative model and discriminative maxmargin classifiers in a unified Bayesian framework. Differently, our model considers both correlations and the relative time distributions between topics rather than the absolute time, which captures richer information of action structures in the complex human activity.…”

Darwintrees for Action Recognition

“…Alfaro et al [1] use a set of pooled key-sequences to quantify relative local intra-and inter-class similarities by projecting the key-sequences to a bank of dictionaries encoding patterns from different temporal positions or action classes. Yang et al [33] jointly learn a high-level representation by combining a hierarchical generative model (that represents actions by distributions over latent spatial temporal patterns) and discriminative max-margin classifiers in a unified Bayesian framework. Fernando et al [6] propose a hierarchical rank pooling -based on [8] -, but on video segments; the first layer performs rank pooling on CNN feature maps and subsequent layers on the result of previous rank pooling operations.…”