Multi‐view human action recognition using 2D motion templates based on MHIs and their HOG description

Abstract

In this paper, a new multi-view human action recognition approach is proposed by exploiting low dimensional motion information of actions. Before feature extraction, pre-processing steps are performed to remove noise from silhouettes, incurred due to imperfect but realistic segmentation. 2D motion templates based on Motion History Image (MHI) are computed for each view/action video which can cope with the high-dimensionality issue, incurred due to multi-camera data. Histograms of Oriented Gradients (HOGs) are used as an efficient description of the MHIs. Finally, a Nearest Neighbor (NN) classifier is employed for the classification of the HOG based description of MHIs. As compared to existing approaches, the proposed method has three advantages: 1) does not require a fixed number of cameras setup during training and testing stages hence missing camera-views can be tolerated, 2) requires less memory and bandwidth requirements and hence 3) is computationally efficient which makes it suitable for real-time action recognition. The proposed method is evaluated on the new MuHAVi-uncut dataset having a large number of action categories and a large set of camera-views with noisy silhouettes. As far as we know, this is the first report of results on that dataset and can be used by future workers as a baseline to improve upon. Experimentation results on multi-view with this dataset gives a high accuracy rate of 95.4% using Leave-One-Sequence-Out (LOSO) cross validation technique and compares well to similar state-of-the-art approaches.

PMHI: Proposals From Motion History Images for Temporal Segmentation of Long Uncut Videos

“…Experiments show that PMHI outperforms the recall rate of recent methods on the MuHAVi-uncut [20] dataset as well as the CVPR 2012 Change Detection dataset [21]. The MuHAVi-uncut is relatively new [22] and thus, to our knowledge, it has not yet been used for temporal segmentation purposes. Therefore, our work can also be used as a baseline for the temporal segmentation of the MuHAVi-uncut dataset.…”

“…MuHAVi-uncut is a dataset of long RGB video recordings (8 cameras) of people doing prescribed actions [22]. The dataset provides a set of silhouettes obtained by a good but not perfect foreground estimation algorithm.…”

Learning a Mid-Level Representation for Multiview Action Recognition

“…Then they utilized histogram of the dominant angle and intensity to represent each sequence and concatenated histograms of all views as the final feature of multiview sequences. Murtaza et al [21] developed a silhouette-based view-independent action recognition scheme. They computed Motion History Images (MHI) for each view and employed Histograms of Oriented Gradients (HOG) to extract low-dimensional description of them.…”

“…Since our method takes humancentered subvolumes recorded from multiple views as input, , , } ∈cub( ),V=1: , =1: ; Predefined parameters dep max , ∈ (0, 1), ∈ (0, 1), and ∈ (0, 1); Output: Decision tree Tree , ; (1) Build a bootstrap dataset , by random sampling from with replacement; (2) Create a root node and set its depth to 1, then assign all cuboids in , to it; (3) Initialize an unsettled node queue Υ = 0 and push the root node into Υ; (4) while Υ ̸ = 0 do (5) Pop the first node in Υ; (6) if depth of is larger than dep max or cuboids assigned to belong to the same action and position then (7) Label node as a leaf, and then calculate P and Q from cuboids at node ; (8) Add a triple ( , , ) into decision tree Tree , ; (9) else (10) Initialize the feature candidate set Δ = 0; (11) if random number < then (12) Add a set of randomly selected optical flow features to Δ; (13) else (14) Add a set of randomly selected HOG3D features to Δ; (15) end if (16) if random number < then (17) Add two-dimensional temporal context features to Δ; (18) end if (19) maxgain = −∞, generate a random number ; (20) for each ∈ Δ do (21) if < then (22) Search for the corresponding threshold and compute information gain ( ) in terms of action labels of cuboids arriving at ; (23) else (24) Search for the corresponding threshold and compute information gain ( ) in terms of positions of cuboids arriving at ; (25) end if (26) if ( ) > maxgain then (27) * = , * = ; (28) end if (29) end for (30) Create left children node and right children node , set their depth to dep + 1, and assign each cuboid arriving at to or according to * and * ; then push node * and * into Υ; (31) Add a quintuple ( , , , * , * ) into decision tree Tree , ; (32) end if (33) end while (34) return Decision tree Tree , ; Algorithm 1: Construction of a decision tree.…”

A Comprehensive Review on Handcrafted and Learning-Based Action Representation Approaches for Human Activity Recognition

“…This motion descriptor is based on motion direction and histogram of motion intensity followed by the support vector machine for classification. Another method based on 2D motion templates using motion history images and histogram of oriented gradients was proposed in [94]. In [50], action recognition method was proposed based on the key elements of motion encoding and local changes in motion direction encoded with the bag-of-words technique.…”