REMOTE: Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos

Ma, Xianzheng; Rahmani, Hossein; Fan, Zhipeng; Yang, Bin; Li, Jun

doi:10.1609/aaai.v36i2.20089

Cited by 5 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to end-to-end 3D pose estimation, the 2D-to-3D pipeline approach divides the task into two independent parts: 2D pose estimation from the image and lifting the 2D pose to a 3D pose. Recent 2D-to-3D lifting methods [6,[19][20][21][22][23] have demonstrated superior performance compared to end-to-end approaches, owing to the reliable and effective 2D keypoint detection methods developed in previous works [24][25][26][27]. To advance 2D-to-3D pose estimation, a Graph Stacked Hourglass Network [22] is introduced.…”

Section: Related Workmentioning

confidence: 99%

A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation

Guo,

Li,

Tan

2024

Applied Sciences

View full text Add to dashboard Cite

The effectiveness of deep neural network models is intricately tied to the distribution of training data. However, in pose estimation, potential discrepancies in root joint positions and inherent variability in biomechanical features across datasets are often overlooked in current training strategies. To address these challenges, a novel Hand Pose Biomechanical Model (HPBM) is developed. In contrast to the traditional 3D coordinate-encoded pose, it provides a more intuitive depiction of the anatomical characteristics of the hand. Through this model, a data normalization approach is implemented to align the root joint and unify the biomechanical features of training samples. Furthermore, the HPBM facilitates a weakly supervised strategy for dataset expansion, significantly enhancing the data diversity. The proposed normalized method is evaluated on two widely used 3D hand pose estimation datasets, RHD and STB, demonstrating superior performance compared to the models trained without normalized datasets. Utilizing ground truth 2D keypoints as input, a reduction of 45.1% and 43.4% in error is achieved on the STB and RHD datasets, respectively. When leveraging 2D keypoints from MediaPipe, a reduction in error by 11.3% and 14.3% is observed on the STB and RHD datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation

Guo,

Li,

Tan

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Ma et al [114] recently proposed a semi-supervised approach that utilizes labeled frames and temporal dynamics (predicted key points) to address the problem of limited availability of temporally sparse annotations in videos. They introduced the REinforced MOtion Transformation nEtwork (REMOTE) framework, where a Motion Transformer (MT) and an RL-based Frame Selection Agent (FSA) are combined.…”

Section: Single Pose Estimation Video-basedmentioning

confidence: 99%

Human Pose Estimation Using Deep Learning: A Systematic Literature Review

Samkari,

Arif,

Alghamdi

et al. 2023

MAKE

View full text Add to dashboard Cite

Human Pose Estimation (HPE) is the task that aims to predict the location of human joints from images and videos. This task is used in many applications, such as sports analysis and surveillance systems. Recently, several studies have embraced deep learning to enhance the performance of HPE tasks. However, building an efficient HPE model is difficult; many challenges, like crowded scenes and occlusion, must be handled. This paper followed a systematic procedure to review different HPE models comprehensively. About 100 articles published since 2014 on HPE using deep learning were selected using several selection criteria. Both image and video data types of methods were investigated. Furthermore, both single and multiple HPE methods were reviewed. In addition, the available datasets, different loss functions used in HPE, and pretrained feature extraction models were all covered. Our analysis revealed that Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are the most used in HPE. Moreover, occlusion and crowd scenes remain the main problems affecting models’ performance. Therefore, the paper presented various solutions to address these issues. Finally, this paper highlighted the potential opportunities for future work in this task.

show abstract

Dancing on the inside: A qualitative study on online dance learning with teacher-AI cooperation

Kang

Yoon

et al. 2023

Educ Inf Technol

View full text Add to dashboard Cite

Recent technologies have extended opportunities for online dance learning by overcoming the limitations of space and time. However, dance teachers report that student–teacher interaction is more likely to be challenging in a distant and asynchronous learning environment than in a conventional dance class, such as a dance studio. To address this issue, we introduce DancingInside, an online dance learning system that encourages a beginner to learn dance by providing timely and sufficient feedback based on Teacher-AI cooperation. The proposed system incorporates an AI-based tutor agent (AI tutor, in short) that uses a 2D pose estimation approach to quantitatively estimate the similarity between a learner's and teacher's performance. We conducted a two-week user study with 11 students and 4 teachers. Our qualitative study results highlight that the AI tutor in DancingInside could support the reflection on a learner's practice and help the performance improvement with multimodal feedback resources. The interview results also reveal that the human teacher's role is essential in complementing the AI feedback. We discuss our design and suggest potential implications for future AI-supported cooperative dance learning systems.

show abstract

REMOTE: Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos

Cited by 5 publications

References 37 publications

A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation

A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation

Human Pose Estimation Using Deep Learning: A Systematic Literature Review

Dancing on the inside: A qualitative study on online dance learning with teacher-AI cooperation

Contact Info

Product

Resources

About