Multi-person Pose Tracking using Sequential Monte Carlo with Probabilistic Neural Pose Predictor

Okada, Masashi; Takenaka, Shinji; Taniguchi, Tadahiro

doi:10.1109/icra40945.2020.9196509

Cited by 5 publications

(3 citation statements)

References 34 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CW-VAE [53], TVA [29], RSSM [26], and SVG [14] mainly focus on image sequence data such as 3D maze [17] and moving handwritten data [55]. [48] and STAF [49] use SVAEs for pose tracking.…”

Section: G Related Workmentioning

confidence: 99%

Ensemble kalman variational objective: a variational inference framework for sequential variational auto-encoders

Ishizone,

Higuchi,

Nakamura

2023

NOLTA

View full text Add to dashboard Cite

Time series model inference can be divided into modeling and optimization. Sequential VAEs have been studied as a modeling technique. As an optimization technique, methods combining variational inference (VI) and sequential Monte Carlo (SMC) have been proposed; however, they have two drawbacks: less particle diversity and biased gradient estimators. This paper proposes Ensemble Kalman Variational Objective (EnKO), a VI framework with the ensemble Kalman filter, to infer latent time-series models. Our proposed method efficiently learns the time-series models because of its particle diversity and unbiased gradient estimators. We demonstrate that our EnKO outperforms previous SMC-based VI methods in the predictive ability for several synthetic and real-world data sets.

show abstract

“…CW-VAE [53], TVA [29], RSSM [26], and SVG [14] mainly focus on image sequence data such as 3D maze [17] and moving handwritten data [55]. [48] and STAF [49] use SVAEs for pose tracking.…”

Section: G Related Workmentioning

confidence: 99%

Ensemble kalman variational objective: a variational inference framework for sequential variational auto-encoders

Ishizone,

Higuchi,

Nakamura

2023

NOLTA

View full text Add to dashboard Cite

show abstract

“…Similarly, Li et al [29] rewrote the weight-sharing CNN as a recurrent network to estimate poses from continuous images. Okada et al [30] designed a recurrent architecture to utilize time-sequence information of poses to manage difficult situations, such as the frequent disappearance and reappearances of poses. Artacho et al [31] proposed UniPose-LSTM adopting a linear sequential LSTM configuration with the waterfall-based approach for temporal human pose estimation in videos.…”

Section: Related Work a Video-based Human Pose Estimationmentioning

confidence: 99%

Learning Human Kinematics by Modeling Temporal Correlations between Joints for Video-based Human Pose Estimation

Dang¹,

Yin²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Estimating human poses from videos is critical in human-computer interaction. By precisely estimating human poses, the robot can provide an appropriate response to the human. Most existing approaches use the optical flow, RNNs, or CNNs to extract temporal features from videos. Despite the positive results of these attempts, most of them only straightforwardly integrate features along the temporal dimension, ignoring temporal correlations between joints. In contrast to previous methods, we propose a plug-and-play kinematics modeling module (KMM) based on the domain-cross attention mechanism to model the temporal correlation between joints across different frames explicitly. Specifically, the proposed KMM models the temporal correlation between any two joints by calculating their temporal similarity. In this way, KMM can learn the motion cues of each joint. Using the motion cues (temporal domain) and historical positions of joints (spatial domain), KMM can infer the initial positions of joints in the current frame in advance. In addition, we present a kinematics modeling network (KIMNet) based on the KMM for obtaining the final positions of joints by combining pose features and initial positions of joints. By explicitly modeling temporal correlations between joints, KIMNet can infer the occluded joints at present according to all joints at the previous moment. Furthermore, the KMM is achieved through an attention mechanism, which allows it to maintain the high resolution of features. Therefore, it can transfer rich historical pose information to the current frame, which provides effective pose information for locating occluded joints. Our approach achieves state-of-the-art results on two standard video-based pose estimation benchmarks. Moreover, the proposed KIMNet shows some robustness to the occlusion, demonstrating the effectiveness of the proposed method.

show abstract

“…Bayesian Neural Networks [20], [21], [22], [23] usually replace the deterministic network's weight parameters with distributions over these parameters, which gives network the ability to estimate the model uncertainty. Given an input, a BNN first samples weights from the weight distributions and then perform the forward calculation, which is always timeconsuming.…”

Section: Bayesian Deep Learningmentioning

confidence: 99%

Probabilistic Human Motion Prediction via A Bayesian Neural Network

Xu¹,

Chen²,

Lan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human motion prediction is an important and challenging topic that has promising prospects in efficient and safe human-robot-interaction systems. Currently, the majority of the human motion prediction algorithms are based on deterministic models, which may lead to risky decisions for robots. To solve this problem, we propose a probabilistic model for human motion prediction in this paper. The key idea of our approach is to extend the conventional deterministic motion prediction neural network to a Bayesian one. On one hand, our model could generate several future motions when given an observed motion sequence. On the other hand, by calculating the Epistemic Uncertainty and the Heteroscedastic Aleatoric Uncertainty, our model could tell the robot if the observation has been seen before and also give the optimal result among all possible predictions. We extensively validate our approach on a large scale benchmark dataset Human3.6m. The experiments show that our approach performs better than deterministic methods. We further evaluate our approach in a Human-Robot-Interaction (HRI) scenario. The experimental results show that our approach makes the interaction more efficient and safer.

show abstract

Multi-person Pose Tracking using Sequential Monte Carlo with Probabilistic Neural Pose Predictor

Cited by 5 publications

References 34 publications

Ensemble kalman variational objective: a variational inference framework for sequential variational auto-encoders

Ensemble kalman variational objective: a variational inference framework for sequential variational auto-encoders

Learning Human Kinematics by Modeling Temporal Correlations between Joints for Video-based Human Pose Estimation

Probabilistic Human Motion Prediction via A Bayesian Neural Network

Contact Info

Product

Resources

About