Human 3D Motion Recognition Based on Spatial-Temporal Context of Joints

Abstract

The paper presents a novel human motion recognition method based on a new form of the Hidden Markov Models, called spatial-temporal hidden markov models (ST-HMM), which can be learnt from a sequence of joints positions. To cope with the high dimensionality of the pose space, in this paper, we exploit the spatial dependency between each pair of spatially connected joints in the articulated skeletal structure, as well as the temporal dependency due to the continuous movement of each of the joints. The spatialtemporal contexts of these joints are learnt from the sequences of joints movements and captured by our ST-HMM. Results of recognizing 11 different action classes on a large number of motion capture sequences as well as synthetic tracking data show that our approach outperforms traditional HMM approach in terms of robustness and recognition rates.

Hybrid generative-discriminative recognition of human action in 3D joint space

“…To validate the proposed 3D action recognition algorithm, we test it on a large 3D MoCap dataset which is obtained from the CMU MoCap database and has been used by [2,7]. Our dataset consists of 8 classes of actions: walk, jog, jump, handshake, punch, climb ladders, sit and wave.…”

“…The confusion matrix of PD-3 is shown in Table. 1 In the second experiment, we evaluate the robustness of our method with respect to the numbers of parts, in comparison with 4 related methods: (1) 2Gram with SVM, (2) HMM with MAP classifier, (3) HMM-Boost [2], (4) ST-HMM with MAP classifier [7]. 2Gram is actually the transition probability matrix of the observed states of a sequences, and serves as the baseline method for sequences recognition.…”

“…As shown in [2,7], exploiting spatial context among joints is useful to recognition. Like [2], we decompose all joints into several overlapped parts (groups).…”

“…Aiming at capturing discriminative information efficiently, we propose to combine HMM with discriminative classifiers based on Posterior Divergence (PD) [1], which is different from ST-HMM [7] or HMM-boost [2]. PD is a principled method to derive explicit feature mapping from generative models, which captures discriminative information of samples by fully exploiting hidden variables.…”

“…In [7], spatial-temporal HMM (ST-HMM) is proposed to model the pair-wise spatial dependency of connected joints. ST-HMM, using maximum a posterior (MAP) classifier, outperforms HMM (also use MAP classifier) as it exploits the additional spatial information.…”