Cross-view human action recognition from depth maps using spectral graph sequences

Abstract

We present a method for view-invariant action recognition from depth cameras based on graph signal processing techniques. Our framework leverages a novel graph representation of an action as a temporal sequence of graphs, onto which we apply a spectral graph wavelet transform for creating our feature descriptor. We evaluate two view-invariant graph types: skeleton-based and keypoint-based. The skeleton-based descriptor captures the spatial pose of the subject, whereas the keypoint-based is able to capture complementary information about human-object interaction and the shape of the point cloud. We investigate the effectiveness of our method by experiments on five publicly available datasets. By the graph structure, our method captures the temporal interaction between depth map interest points and achieves a 19.8% increase in performance compared to state-of-the-art results for cross-view action recognition, and competing results for frontal-view action recognition and human-object interaction. Namely, our method results in 90.8% accuracy on the cross-view N-UCLA Multiview Action3D dataset and 91.4% accuracy on the challenging MSRAction3D dataset in the cross-subject setting. For human-object interaction, our method achieves 72.3% accuracy on the Online RGBD Action dataset. We also achieve 96.0% and 98.8% accuracy on the MSRActionPairs3D and UCF-Kinect datasets, respectively.

Linear Time Additively Exact Algorithm for Transformation of Chain-Cycle Graphs for Arbitrary Costs of Deletions and Insertions

“…Chains and cycles are also used to represent parts that have long or compact forms and to describe the frame of an area or human pose, among other examples. Among many such applications, which are not related to genomics, let us mention, as an example, [17].…”

Learning Human Pose Models from Synthesized Data for Robust RGB-D Action Recognition

“…This method utilizes the cross-modality and private information, which are also de-noised before the final classification. Kerola et al (2017) used spatio-temporal key points (STKP) and skeletons to represent an action as a temporal sequence of graphs, and then applied the spectral graph wavelet transform to create the action descriptors. Varol et al (2017b) recently proposed SUR- REAL to synthesize human pose images for the task of body segmentation and depth estimation.…”

Novel Cross-View Human Action Model Recognition Based on the Powerful View-Invariant Features Technique

“…The authors of [16] also introduced a 3D trajectory which can describe the action better; however, it does not emphasize the percentage of contribution of each view to that trajectory. Kerola et al [17] took a temporal sequence of graphs as an illustration of a graph representation action and used that to create a feature descriptor by applying a spectral graph wavelet transform. The authors of [17] also emphasized two well-known types of view-invariant graphs-key point-based and skeleton-based graphs.…”

“…Kerola et al [17] took a temporal sequence of graphs as an illustration of a graph representation action and used that to create a feature descriptor by applying a spectral graph wavelet transform. The authors of [17] also emphasized two well-known types of view-invariant graphs-key point-based and skeleton-based graphs. Rahmani et al [18] devised a histogram of oriented principal components (HOPC) that is robust to noise.…”