A Two-Stream Recurrent Network for Skeleton-based Human Interaction Recognition

Men, Qianhui; Ho, Edmond S. L.; Shum, Hubert P. H.; Leung, Howard

doi:10.1109/icpr48806.2021.9412538

Cited by 5 publications

(1 citation statement)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li and Leung [23] applied a multiple kernel learning method to an interaction graph constructed from the relative variance of joint relative distances. Two-stream RNNs are proposed in [37,57] where interactions between two persons are modeled by concatenating the 3D coordinates of their corresponding joints, or by augmenting the input sequence with distances between their joints. In [46], Relational Network [51] is extended to automatically infer intra-person and inter-person joint relationships.…”

Section: Two-person Interaction Recognition From 3d Skeleton Sequencesmentioning

confidence: 99%

GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition

Nguyen¹

2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a novel method for representation and classification of two-person interactions from 3D skeleton sequences. The key idea of our approach is to use Gaussian distributions to capture statistics on R n and those on the space of symmetric positive definite (SPD) matrices. The main challenge is how to parametrize those distributions. Towards this end, we develop methods for embedding Gaussian distributions in matrix groups based on the theory of Lie groups and Riemannian symmetric spaces. Our method relies on the Riemannian geometry of the underlying manifolds and has the advantage of encoding highorder statistics from 3D joint positions. We show that the proposed method achieves competitive results in two-person interaction recognition on three benchmarks for 3D human activity understanding.

show abstract

Section: Two-person Interaction Recognition From 3d Skeleton Sequencesmentioning

confidence: 99%

GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition

Nguyen¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

PyTorch-based implementation of label-aware graph representation for multi-class trajectory prediction

Men

Shum

2022

Software Impacts

Self Cite

View full text Add to dashboard Cite

Trajectory Prediction under diverse patterns has attracted increasing attention in multiple real-world applications ranging from urban traffic analysis to human motion understanding, among which graph convolution network (GCN) is frequently adopted with its superior ability in modeling the complex trajectory interactions among multiple humans. In this work, we propose a python package by enhancing GCN with class label information of the trajectory, such that we can explicitly model not only human trajectories but also that of other road users such as vehicles. This is done by integrating a label-embedded graph with the existing graph structure in the standard graph convolution layer. The flexibility and the portability of the package also allow researchers to employ it under more general multi-class sequential prediction tasks.

show abstract

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

Phuong,

Tran,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Graph Convolutional Networks (GCNs) have gained widespread adoption in modeling human skeleton sequences for two-person interaction recognition. Most GCN-based models achieve state-of-the-art results by leveraging either intra-body or inter-body connections. However, using only intra-body relations may ignore important interactive features between two individuals, whereas relying on inter-body relations may weaken the specific motion dynamics of each skeleton. To address these shortcomings, we propose a Distinct Motion-Preserving GCN (DMP-GCN) that utilizes intra-body and inter-body graphs to extract interactive features from two human bodies while preserving the distinct motion characteristics of each skeleton. Specifically, two motion-specific streams are adopted to capture specific motion features of each human skeleton and an interactive stream is applied to model the interactive dynamics of two bodies. In addition, we introduce a new graph labeling strategy called Distance Variation Labeling which is a datadriven approach for defining the edge strength in the skeleton graph. Extensive experiments show that our proposed approaches outperform state-of-the-art methods on two large-scale human interaction datasets, NTU RGB+D (mutual) and NTU RGB+D 120 (mutual).

show abstract

A Two-Stream Recurrent Network for Skeleton-based Human Interaction Recognition

Cited by 5 publications

References 53 publications

GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition

GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition

PyTorch-based implementation of label-aware graph representation for multi-class trajectory prediction

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

Contact Info

Product

Resources

About