Action recognition using global spatio-temporal features derived from sparse representations

Abstract

Recognizing actions is one of the important challenges in computer vision with respect to video data, with applications to surveillance, diagnostics of mental disorders, and video retrieval. Compared to other data modalities such as documents and images, processing video data demands orders of magnitude higher computational and storage resources. One way to alleviate this difficulty is to focus the computations to informative (salient) regions of the video. In this paper, we propose a novel global spatio-temporal selfsimilarity measure to score saliency using the ideas of dictionary learning and sparse coding. In contrast to existing methods that use local spatio-temporal feature detectors along with descriptors (such as HOG, HOG3D, HOF, etc.), dictionary learning helps consider the saliency in a global setting (on the entire video) in a computationally efficient way. We consider only a small percentage of the most salient (least self-similar) regions found using our algorithm, over which spatio-temporal descriptors such as HOG and region covariance descriptors are computed. The ensemble of such block descriptors in a bag-of-features framework provides a holistic description of the motion sequence which can be used in a classification setting. Experiments on several benchmark datasets in video based action classification demonstrate that our approach performs competitively to the state of the art.

Review on Vision based Human Activity Analysis

“…Extraction of Salient region: In order to extract salient/important features some methods like commonly used are STIP method [8], sparse coding [7] and STFT. Here no extraction of human area done.…”

Gated spatio and temporal convolutional neural network for activity recognition: towards gated multimodal deep learning

“…However, how to fuse or integrate all streams is still an open question. Before deeply learned features became popular, there were many research approaches to video classification using various methods, especially handcrafted methods such as spatiotemporal features [1], dense trajectories [9], and local autocorrelation [19]. Three-dimensional (3D) CNN was the first attempt to train spatiotemporal features for video classification using deep CNNs.…”

“…1 Graduate School of Information Engineering, Hiroshima University, Higashi Hiroshima, Japan. 2 Department of Information Engineering, Hiroshima University, Higashi Hiroshima, Japan.…”

“…Video classification using CNN has achieved significant improvement since the use of a collection of still images and ImageNet weights to be fine tuned on two stream *Correspondence: cbasemaster@gmail.com 1 Graduate School of Information Engineering, Hiroshima University, Higashi Hiroshima, Japan Full list of author information is available at the end of the article network. In this paper, we implemented the two-stream CNN proposed by Simonyan [11] for human action recognition, which uses spatial and motion streams using the Chainer framework [13].…”

RGBD Human Action Recognition using Multi-Features Combination and K-Nearest Neighbors Classification

“…We are using the seven translation, After the two feature sets are computed, the feature vectors are combined and encoded into a single code by using the bag of features algorithm [13]. The unsupervised learning K-means clustering and supervised learning K-nearest neighbor (KNN) are used for classification the different action from videos.…”