Fusing $${\mathcal {R}}$$ R Features and Local Features with Context-Aware Kernels for Action Recognition

Abstract

The performance of action recognition in video sequences depends significantly on the representation of actions and the similarity measurement between the representations. In this paper, we combine two kinds of features extracted from the spatio-temporal interest points with context-aware kernels for action recognition. For the action representation, local cuboid features extracted around interest points are very popular using a Bag of Visual Words (BOVW) model. Such representations, however, ignore potentially valuable information about the global spatio-temporal distribution of interest points. We propose a new global feature to capture the detailed geometrical distribution of interest points. It is calculated by using the 3D R transform which is defined as an extended 3D discrete Radon transform, followed by the application of a two-directional two-dimensional principal component analysis. For the similarity measurement, we model a video set as an optimized probabilistic hypergraph and propose a context-aware kernel to measure high order relationships among videos. The context-aware kernel is more robust to the noise and outliers in the data than the traditional context-free kernel which just considers the pairwise relationships between videos. The hyperedges of the hypergraph are constructed based on a learnt Mahalanobis distance metric. Any disturbing information from other classes is excluded from each hyperedge. Finally, a multiple kernel learning algorithm is designed by integrating the l 2 norm regularization into a linear SVM classifier to fuse the R feature and the BOVW representation for action recognition. Experimental results on several datasets demonstrate the effectiveness of the proposed approach for action recognition.

Insights on Research-based Approaches in Human Activity Recognition System

“…The outcome of this study shows that it had good ability that understands an action from unseen views. In the same way the work of Yuan et al [42], the authors have used the multiple techniques which including R-features transform with the context-aware kernel learning algorithm that captures the distributed actions and to classify the similarities between the representations of activity in the video sequence. The experimental performance delivers that the proposed approach is effective for human action recognition from the captured videos.…”

A Novel Parameter Initialization Technique Using RBM-NN for Human Action Recognition

“…In the sequential method, the temporal features such as appearance and pose are obtained from the hidden Markov model [ 54 – 56 ], conditional random fields [ 57 – 60 ], and structured support vector machine [ 61 – 64 ]. Furthermore, representative key poses are learned for efficient representation of human actions [ 33 , 34 , 65 – 72 ] to build a compact pose sequence.…”

“…Deep learning techniques [ 73 ] such as 2D ConvNets [ 21 , 74 ] and 3D ConvNets [ 26 ] perform feature learning via convolution operator and temporal modeling [ 75 ]. The initialization of a deep neural network [ 72 ] is crucial for training the model. To ensure that the state of the hidden layers follow a uniform distribution, a model parameter [ 76 – 78 ] is initialized.…”