Multiple Batches of Motion History Images (MB-MHIs) for Multi-view Human Action Recognition

Abstract

The recognition of human actions recorded in a multi-camera environment faces the challenging issue of viewpoint variation. Multiview methods employ videos from different views to generate a compact view-invariant representation of human actions. This paper proposes a novel multi-view human action recognition approach that uses multiple low-dimensional temporal templates and a reconstruction-based encoding scheme. The proposed approach is based upon the extraction of multiple 2D Motion History Images (MHIs) of human action videos over nonoverlapping temporal windows, constructing Multiple Batches of Motion History Images (MB-MHIs). Then, two kinds of descriptions are computed for these MHIs batches based on (1) a deep Residual Network (ResNet) and (2) Histogram of Oriented Gradients (HOG) to effectively quantify a change in gradient. ResNet descriptions are average pooled at each batch. HOG descriptions are processed independently at each batch to learn a class-based dictionary using a K-Spectral Value Decomposition algorithm (K-SVD). Later, the sparse codes of feature descriptions are obtained using an Orthogonal Matching Pursuit (OMP) approach. These sparse codes are average pooled to extract encoded feature vectors. Then, encoded feature vectors at each batch are fused to form a final view-invariant feature representation. Finally, a linear Support Vector Machine (SVM) classifier is trained for action recognition. Experimental results are given on three versions of a multi-view dataset: MuHAVi-8, MuHAVi-14, and MuHAViuncut. The proposed approach shows promising results when tested for a novel camera. Results on deep features indicate that action representation by MB-MHIs is more view-invariant than single MHIs.

Motion Recognition Based on Sum of the Squared Errors Distribution

“…By fitting this curve, we get the fitting formula shown in (7). The limit of SSE is calculated in (8) Set σ be 5% of L. Then σ=0.32, and we get = 10 according to (7) and (9). From the experimental results in Table Ⅰ, we can obtain that when k≥10, (9) is satisfied.…”

“…For instance, Tewari designed a weak model of gesture recognition based on five postures in 2017 to solve the problem of non-contact interaction in cars [4]. Lu designed a smart wheelchair motion control system based on gesture recognition for people with physical disabilities [5], In recent years, motion recognition technology based on vision has been rapidly developed, such as flow-guided (FG) dynamic maps [6], Residual Network (ResNet) [7], Multiple Batches of Motion History Images (MB-MHIs) [8], skeleton edge motion networks (SEMN) [9],deep convolutional neural network (DCNN) [10], and Random forest (RF) [11]. Most of these methods use image or video information obtained by cameras to perform human motion recognition.…”

A Review on Computer Vision-Based Methods for Human Action Recognition

“…Figure 4 illustrates the accuracy variations of the recognition model over different components. [192].…”

“…A CNN model was then proposed to perform feature learning from the multi-view dynamic images. Multiple batches of motion history images (MB-MHIs) have been constructed by [ 192 ]. This information is then used to compute two descriptors by using: a deep residual network (ResNet) and histogram of oriented gradients (HOG).…”

“…Figure 4. The accuracy variations with the number of frames and number of batches as mentioned in[192].…”