Unsupervised Representation Learning With Long-Term Dynamics for Skeleton Based Action Recognition

Zheng, Nenggan; Wen, Jun; Liu, Risheng; Long, Liangqu; Dai, Jianhua; Gong, Zhefeng

doi:10.1609/aaai.v32i1.11853

Cited by 92 publications

(52 citation statements)

References 23 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Linear Evaluation Results on NTU-60. As shown in Table 4, for a single stream (i.e., joint stream), our AimCLR outperforms all other methods (Zheng et al 2018;Lin et al 2020;Rao et al 2021;Su, Liu, and Shlizerman 2020;Nie, Liu, and Liu 2020;Li et al 2021). For the performance of the Table 8: Finetuned results on NTU-60 and NTU-120 dataset. "…”

Section: Comparison With State-of-the-artmentioning

confidence: 88%

“…Self-supervised Skeleton-based Action Recognition. LongT GAN (Zheng et al 2018) proposes to use the encoder-decoder to regenerate the input sequence to obtain useful feature representation. P&C (Su, Liu, and Shlizerman 2020) proposes a training strategy to weaken the decoder, forcing the encoder to learn more discriminative features.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, several works (Zheng et al 2018;Su, Liu, and Shlizerman 2020;Lin et al 2020) focus on designing pretext tasks for self-supervised methods to learn action representations from unlabeled skeleton data. With the development of contrastive self-supervised learning and its ability to make feature representations have better discrimination, several works (Rao et al 2021;Li et al 2021) directly rely on the contrastive learning framework, using normal augmentations to construct similar positive samples.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition

Guo

Liu

Chen

et al. 2022

AAAI

View full text Add to dashboard Cite

In recent years, self-supervised representation learning for skeleton-based action recognition has been developed with the advance of contrastive learning methods. The existing contrastive learning methods use normal augmentations to construct similar positive samples, which limits the ability to explore novel movement patterns. In this paper, to make better use of the movement patterns introduced by extreme augmentations, a Contrastive Learning framework utilizing Abundant Information Mining for self-supervised action Representation (AimCLR) is proposed. First, the extreme augmentations and the Energy-based Attention-guided Drop Module (EADM) are proposed to obtain diverse positive samples, which bring novel movement patterns to improve the universality of the learned representations. Second, since directly using extreme augmentations may not be able to boost the performance due to the drastic changes in original identity, the Dual Distributional Divergence Minimization Loss (D3M Loss) is proposed to minimize the distribution divergence in a more gentle way. Third, the Nearest Neighbors Mining (NNM) is proposed to further expand positive samples to make the abundant information mining process more reasonable. Exhaustive experiments on NTU RGB+D 60, PKU-MMD, NTU RGB+D 120 datasets have verified that our AimCLR can significantly perform favorably against state-of-the-art methods under a variety of evaluation protocols with observed higher quality action representations. Our code is available at https://github.com/Levigty/AimCLR.

show abstract

Section: Comparison With State-of-the-artmentioning

confidence: 88%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition

Guo

Liu

Chen

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…To classify activities, encoded features were given to the KNN classifier. Zheng et al [33] extracted deep features for classifying actions. They introduced a GAN autoencoder and extracted the dynamic motions from skeleton frames.…”

Section: Human Activity Recognition and Discoverymentioning

confidence: 99%

Flexible Multi-Objective Particle Swarm Optimization Clustering with Game Theory to Address Human Activity Recognition Fully Unsupervised

Hadikhani¹,

Lai²,

Ong³

2022

Preprint

View full text Add to dashboard Cite

Most research in human activity recognition is supervised, while non-supervised approaches are not completely unsupervised. In this paper, we provide a novel flexible multi-objective particle swarm optimization (PSO) clustering method based on game theory (FMOPG) to discover human activities fully unsupervised. Unlike conventional clustering methods that estimate the number of clusters and are very time-consuming and inaccurate, an incremental technique is introduced which makes the proposed method flexible in dealing with the number of clusters. Using this technique, clusters that have a better connectedness and good separation from other clusters are gradually selected. To improve the convergence speed of PSO in achieving the best solution and dealing with spherical shape clusters, updating of particles' velocity is modified using the concept of mean-shift vector. To solve multi-objective optimization problems, Nash equilibrium in game theory is used to select the optimal solution on the pareto front. Gaussian mutation is also employed on the pareto front to generate diverse solutions and create a balance between exploitation and exploration. The proposed method is compared with state-of-the-art methods on five challenging datasets. FMOPG has improved clustering accuracy by 3.65% compared to automated methods. Moreover, the incremental technique has improved the clustering time by 71.18%.

show abstract

“…It impels the exploration of learning skeleton-based action representation in an unsupervised manner [15,24,30,14]. Often unsupervised methods use pretext tasks to generate the supervision signals, such as reconstruction [7,44], autoregression [12,30] and jigsaw puzzles [22,36]. Consequently, the learning highly relies on the quality of the designed pretext tasks, and those tasks are hard to be generalized for different downstream tasks.…”

Section: Introductionmentioning

confidence: 99%

Contrastive Positive Mining for Unsupervised 3D Action Representation Learning

Zhang¹,

Hou²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent contrastive based 3D action representation learning has made great progress. However, the strict positive/negative constraint is yet to be relaxed and the use of non-self positive is yet to be explored. In this paper, a Contrastive Positive Mining (CPM) framework is proposed for unsupervised skeleton 3D action representation learning. The CPM identifies non-self positives in a contextual queue to boost learning. Specifically, the siamese encoders are adopted and trained to match the similarity distributions of the augmented instances in reference to all instances in the contextual queue. By identifying the non-self positive instances in the queue, a positive-enhanced learning strategy is proposed to leverage the knowledge of mined positives to boost the robustness of the learned latent space against intra-class and inter-class diversity. Experimental results have shown that the proposed CPM is effective and outperforms the existing state-of-the-art unsupervised methods on the challenging NTU and PKU-MMD datasets.

show abstract

Unsupervised Representation Learning With Long-Term Dynamics for Skeleton Based Action Recognition

Cited by 92 publications

References 23 publications

Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition

Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition

Flexible Multi-Objective Particle Swarm Optimization Clustering with Game Theory to Address Human Activity Recognition Fully Unsupervised

Contrastive Positive Mining for Unsupervised 3D Action Representation Learning

Contact Info

Product

Resources

About