Efficient Temporal Sequence Comparison and Classification Using Gram Matrix Embeddings on a Riemannian Manifold

Zhang, Xikang; Wang, Yin; Gou, Mengran; Sznaier, Mario; Camps, Octavia

doi:10.1109/cvpr.2016.487

Cited by 72 publications

(76 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, we evaluated the proposed metric with respect to other metrics used in state of the art solutions. Specifically, given two matrices G 1 and G 2 in S + (3, n), we compared our results with two other possible metrics: (1) as proposed in [12], [59], we used d Pn that was defined in Eq. (7) to compare G 1 and G 2 by regularizing their ranks, i.e., making them n full-rank, and considering them in P n (the space of n-by-n positive definite matrices), d Pn (G 1 , G 2 ) = d Pn (G 1 + I n , G 2 + I n ); (2) we used the Euclidean flat distance d F + (G 1 , G 2 ) = G 1 − G 2 F , where .…”

Section: Resultsmentioning

confidence: 99%

“…In both of these works kernelized versions of covariance matrices are considered. Zhang et al [12] represented temporal landmark sequences using regularized Gram matrices derived from the Hankel matrices of landmark sequences. The authors show that the Hankel matrix of a 3D landmark sequence is related to an Auto-Regressive (AR) model [13], where only the linear relationships between landmark static observations are captured.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

Kacem

Daoudi

Amor

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

In this paper, we propose a novel space-time geometric representation of human landmark configurations and derive tools for comparison and classification. We model the temporal evolution of landmarks as parametrized trajectories on the Riemannian manifold of positive semidefinite matrices of fixed-rank. Our representation has the benefit to bring naturally a second desirable quantity when comparing shapes -the spatial covariance -in addition to the conventional affine-shape representation. We derived then geometric and computational tools for rate-invariant analysis and adaptive re-sampling of trajectories, grounding on the Riemannian geometry of the underlying manifold. Specifically, our approach involves three steps: (1) landmarks are first mapped into the Riemannian manifold of positive semidefinite matrices of fixed-rank to build time-parameterized trajectories; (2) a temporal warping is performed on the trajectories, providing a geometry-aware (dis-)similarity measure between them; (3) finally, a pairwise proximity function SVM is used to classify them, incorporating the (dis-)similarity measure into the kernel function. We show that such representation and metric achieve competitive results in applications as action recognition and emotion recognition from 3D skeletal data, and facial expression recognition from videos. Experiments have been conducted on several publicly available up-to-date benchmarks.In this document, we provide the mathematical preliminaries to our approach. We also give further details on the conducted experiments and the reported results.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

Kacem

Daoudi

Amor

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Accuracy (%) Moving Pose [40] 56.34 JOULE-pose [10] 74.60 HBRNN [5] 77.40 TF [6] 80.69 Lie Group [37] 82.69 Gram Matrix [42] 85. 39 tions and the TTN outputs, it can be readily observed that the TTN performs alignment of the sequences which then makes the classification problem much easier.…”

Section: Methodsmentioning

confidence: 99%

“…The results obtained are shown in Table 1. In addition to our experiments, we have reported results given in [7] for other important algorithms used for 3D pose-based action recognition including JOULE-pose [10], Moving Pose [40], Hierarchical Recurrent Neural Networks (HBRNN) [5], Transition Forests (TF) [6], and Lie Groups [37] and the Gram Matrix method [42], the last two of which also used DTW for sequence alignment, as well as non-Euclidean features to help improve performance. Among the baseline neural networks, TCN-32 led to the best results for this dataset, and addition of more layers did not yield better performance.…”

Section: Icl First-person Hand Action Datasetmentioning

confidence: 99%

Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping

Lohit

Wang

Turaga

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Many time-series classification problems involve developing metrics that are invariant to temporal misalignment. In human activity analysis, temporal misalignment arises due to various reasons including differing initial phase, sensor sampling rates, and elastic time-warps due to subjectspecific biomechanics. Past work in this area has only looked at reducing intra-class variability by elastic temporal alignment. In this paper, we propose a hybrid modelbased and data-driven approach to learn warping functions that not just reduce intra-class variability, but also increase inter-class separation. We call this a temporal transformer network (TTN). TTN is an interpretable differentiable module, which can be easily integrated at the front end of a classification network. The module is capable of reducing intra-class variance by generating input-dependent warping functions which lead to rate-robust representations. At the same time, it increases inter-class variance by learning warping functions that are more discriminative. We show improvements over strong baselines in 3D action recognition on challenging datasets using the proposed framework. The improvements are especially pronounced when training sets are smaller.

show abstract

“…We follow usual training and testing splits proposed in the literature. For Florence3D, G3D, and UTKinect, we use the protocols of [7], [33], [34]. For MSR-Action3D, we adopt the splits originally proposed by [44].…”

Section: D Action Recognition Datasets and Preprocessingmentioning

confidence: 99%

Scalable and compact 3D action recognition with approximated RBF kernel machines

Cavazza

Morerio

Murino

2019

Pattern Recognition

View full text Add to dashboard Cite

Despite the recent deep learning (DL) revolution, kernel machines still remain powerful methods for action recognition. DL has brought the use of large datasets and this is typically a problem for kernel approaches, which are not scaling up efficiently due to kernel Gram matrices. Nevertheless, kernel methods are still attractive and more generally applicable since they can equally manage different sizes of the datasets, also in cases where DL techniques show some limitations. This work investigates these issues by proposing an explicit approximated representation that, together with a linear model, is an equivalent, yet scalable, implementation of a kernel machine. Our approximation is directly inspired by the exact feature map that is induced by an RBF Gaussian kernel but, unlike the latter, it is finite dimensional and very compact. We justify the soundness of our idea with a theoretical analysis which proves the unbiasedness of the approximation, and provides a vanishing bound for its variance, which is shown to decrease much rapidly than in alternative methods in the literature. In a broad experimental validation, we assess the superiority of our approximation in terms of 1) ease and speed of training, 2) compactness of the model, and 3) improvements with respect to the state-of-the-art performance.

show abstract

Efficient Temporal Sequence Comparison and Classification Using Gram Matrix Embeddings on a Riemannian Manifold

Cited by 72 publications

References 33 publications

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping

Scalable and compact 3D action recognition with approximated RBF kernel machines

Contact Info

Product

Resources

About