Learning a Mid-Level Representation for Multiview Action Recognition

Abstract

Recognizing human actions in videos is an active topic with broad commercial potentials. Most of the existing action recognition methods are supposed to have the same camera view during both training and testing. And thus performances of these single-view approaches may be severely influenced by the camera movement and variation of viewpoints. In this paper, we address the above problem by utilizing videos simultaneously recorded from multiple views. To this end, we propose a learning framework based on multitask random forest to exploit a discriminative mid-level representation for videos from multiple cameras. In the first step, subvolumes of continuous human-centered figures are extracted from original videos. In the next step, spatiotemporal cuboids sampled from these subvolumes are characterized by multiple low-level descriptors. Then a set of multitask random forests are built upon multiview cuboids sampled at adjacent positions and construct an integrated mid-level representation for multiview subvolumes of one action. Finally, a random forest classifier is employed to predict the action category in terms of the learned representation. Experiments conducted on the multiview IXMAS action dataset illustrate that the proposed method can effectively recognize human actions depicted in multiview videos.

Pairwise-Covariance Multi-view Discriminant Analysis for Robust Cross-View Human Action Recognition

“…Cross-view Multi-view Liu et al [9] 76.32 N/A Zheng et al [38] 95.1 99.32 Zheng et al [52] 97.8 99.4 Ulhaq et al [53] 66.82 92.47 Zhang et al [54] 84.1 N/A Liu et al [55] N/A 90. Table 6 compares our best combination of ResNet50-3D and pc-MvDA with state-of-the-art frameworks.…”

Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning

“…To make the gradient calculation primarily depend on the classification loss of the source domain data in the early stage of training, the initial value of K is set to a small value of 0. Advances in Multimedia as Equation ( 6) and the optimization objective is defined in Equation (7).…”

“…This approach has achieved remarkable results in various tasks, such as image classification, target detection, and image segmentation, with notable improvements in accuracy and speed [2][3][4][5][6]. Deeplearning-based methods have become the dominant approach in the field of human action recognition [7,8].…”