Recent Advances of Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective

Liu, Wu; Bao, Qing; Sun, Yu; Mei, Tao

doi:10.1145/3524497

Cited by 82 publications

(26 citation statements)

References 192 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…After the pose estimation and bounding box prediction, we obtain the necessary prior information to be used in monocular tracking and inter-camera ReID. There are many popular methods for monocular pedestrian tracking that design ingenious structures of neural networks to deal with occlusions or some other challenges in the video, pursuing high monocular tracking performance [ 3 , 5 ].…”

Section: Methods Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

Hachiuma

Saitō

et al. 2022

J. Imaging

View full text Add to dashboard Cite

Multi-camera multi-person (MCMP) tracking and re-identification (ReID) are essential tasks in safety, pedestrian analysis, and so on; however, most research focuses on outdoor scenarios because they are much more complicated to deal with occlusions and misidentification in a crowded room with obstacles. Moreover, it is challenging to complete the two tasks in one framework. We present a trajectory-based method, integrating tracking and ReID tasks. First, the poses of all surgical members captured by each camera are detected frame-by-frame; then, the detected poses are exploited to track the trajectories of all members for each camera; finally, these trajectories of different cameras are clustered to re-identify the members in the operating room across all cameras. Compared to other MCMP tracking and ReID methods, the proposed one mainly exploits trajectories, taking texture features that are less distinguishable in the operating room scenario as auxiliary cues. We also integrate temporal information during ReID, which is more reliable than the state-of-the-art framework where ReID is conducted frame-by-frame. In addition, our framework requires no training before deployment in new scenarios. We also created an annotated MCMP dataset with actual operating room videos. Our experiments prove the effectiveness of the proposed trajectory-based ReID algorithm. The proposed framework achieves 85.44% accuracy in the ReID task, outperforming the state-of-the-art framework in our operating room dataset.

show abstract

Section: Methods Frameworkmentioning

confidence: 99%

“…To estimate the locations and movements of staff, we need to track everyone in the room and detect their poses [ 3 , 4 , 5 , 6 ]. Hassaballah et al [ 7 ] proposed a robust vehicle detection and tracking approach using a multi-scale deep convolution neural network.…”

Section: Introductionmentioning

confidence: 99%

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

Hachiuma

Saitō

et al. 2022

J. Imaging

View full text Add to dashboard Cite

show abstract

“…Layout GAN. In order to represent the human pose, we first use the human keypoint detection method [3,24,28,29] to predict the 2-D keypoint 𝐾 ∈ R 2×25 of the human body. Then according to the predefined connection strategy, we can get the pose connection map 𝑃 𝑡 𝑑 ∈ R 3×𝐻 ×𝑊 of the driving person, where 𝐻 × 𝑊 is the resolution of the image.…”

Section: Region Generation Networkmentioning

confidence: 99%

REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfer

Yang

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Human Video Motion Transfer (HVMT) aims to, given an image of a source person, generate his/her video that imitates the motion of the driving person. Existing methods for HVMT mainly exploit Generative Adversarial Networks (GANs) to perform the warping operation based on the flow estimated from the source person image and each driving video frame. However, these methods always generate obvious artifacts due to the dramatic differences in poses, scales, and shifts between the source person and the driving person. To overcome these challenges, this paper presents a novel REgionto-whole human MOtion Transfer (REMOT) framework based on GANs. To generate realistic motions, the REMOT adopts a progressive generation paradigm: it first generates each body part in the driving pose without flow-based warping, then composites all parts into a complete person of the driving motion. Moreover, to preserve the natural global appearance, we design a Global Alignment Module to align the scale and position of the source person with those of the driving person based on their layouts. Furthermore, we propose a Texture Alignment Module to keep each part of the person aligned according to the similarity of the texture. Finally, through extensive quantitative and qualitative experiments, our REMOT achieves state-of-the-art results on two public benchmarks.

show abstract

“…Detecting human poses, for example, is critical for intelligent robots to adjust their actions and allocate their attention properly when interacting with humans [1]. Moreover, HPE has been widely exploited for many computer vision tasks, such as action recognition [2], [3], virtual reality [4], and human-computer interaction [5]- [7]. According to the data source type, HPE can be roughly divided into two scopes, i.e., image-based HPE and video-based HPE.…”

Section: Introductionmentioning

confidence: 99%

Learning Human Kinematics by Modeling Temporal Correlations between Joints for Video-based Human Pose Estimation

Dang¹,

Yin²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Estimating human poses from videos is critical in human-computer interaction. By precisely estimating human poses, the robot can provide an appropriate response to the human. Most existing approaches use the optical flow, RNNs, or CNNs to extract temporal features from videos. Despite the positive results of these attempts, most of them only straightforwardly integrate features along the temporal dimension, ignoring temporal correlations between joints. In contrast to previous methods, we propose a plug-and-play kinematics modeling module (KMM) based on the domain-cross attention mechanism to model the temporal correlation between joints across different frames explicitly. Specifically, the proposed KMM models the temporal correlation between any two joints by calculating their temporal similarity. In this way, KMM can learn the motion cues of each joint. Using the motion cues (temporal domain) and historical positions of joints (spatial domain), KMM can infer the initial positions of joints in the current frame in advance. In addition, we present a kinematics modeling network (KIMNet) based on the KMM for obtaining the final positions of joints by combining pose features and initial positions of joints. By explicitly modeling temporal correlations between joints, KIMNet can infer the occluded joints at present according to all joints at the previous moment. Furthermore, the KMM is achieved through an attention mechanism, which allows it to maintain the high resolution of features. Therefore, it can transfer rich historical pose information to the current frame, which provides effective pose information for locating occluded joints. Our approach achieves state-of-the-art results on two standard video-based pose estimation benchmarks. Moreover, the proposed KIMNet shows some robustness to the occlusion, demonstrating the effectiveness of the proposed method.

show abstract

Recent Advances of Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective

Cited by 82 publications

References 192 publications

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfer

Learning Human Kinematics by Modeling Temporal Correlations between Joints for Video-based Human Pose Estimation

Contact Info

Product

Resources

About