Socially and Contextually Aware Human Motion and Pose Forecasting

Adeli, Vida; Adeli, Ehsan; Reid, Ian R.; Niebles, Juan Carlos; Rezatofighi, Hamid

doi:10.48550/arxiv.2007.06843

Cited by 2 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GRUs are often used within encoder and decoder networks [32], [56] to extract motion features and predict future frames in a motion synthesis context. They have been used to forecast context-aware motions, such as modelling motions driven by interactions with other humans or objects [1], [10].…”

Section: B Recurrent Modelsmentioning

confidence: 99%

Adversarial Attention for Human Motion Synthesis

Malek-Podjaski¹,

Deligianni²

2022

Preprint

View full text Add to dashboard Cite

Analysing human motions is a core topic of interest for many disciplines, from Human-Computer Interaction, to entertainment, Virtual Reality and healthcare. Deep learning has achieved impressive results in capturing human pose in realtime. On the other hand, due to high inter-subject variability, human motion analysis models often suffer from not being able to generalise to data from unseen subjects due to very limited specialised datasets available in fields such as healthcare. However, acquiring human motion datasets is highly timeconsuming, challenging, and expensive. Hence, human motion synthesis is a crucial research problem within deep learning and computer vision. We present a novel method for controllable human motion synthesis by applying attention-based probabilistic deep adversarial models with end-to-end training. We show that we can generate synthetic human motion over both short-and long-time horizons through the use of adversarial attention. Furthermore, we show that we can improve the classification performance of deep learning models in cases where there is inadequate real data, by supplementing existing datasets with synthetic motions.

show abstract

Section: B Recurrent Modelsmentioning

confidence: 99%

Adversarial Attention for Human Motion Synthesis

Malek-Podjaski¹,

Deligianni²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The existence of multiple valid futures for a given observed sequence is a crucial attribute of several forecasting tasks, especially surrounding the dynamics of low-level human behavior. These tasks include the forecasting of trajectories of pedestrians [1][2][3][4][5], vehicles [6][7][8][9], and autonomous robots [10,11], or other more general nonverbal cues of humans [12][13][14][15][16] and artificial virtual agents [17] in group conversation settings. Consequently, rather than making point predictions, several machine learning methods in these settings have attempted to forecast a distribution over plausible futures [5,12].…”

Section: Introductionmentioning

confidence: 99%

Why Did This Model Forecast This Future? Closed-Form Temporal Saliency Towards Causal Explanations of Probabilistic Forecasts

Raman¹,

Hung²,

Loog³

2022

Preprint

View full text Add to dashboard Cite

Forecasting tasks surrounding the dynamics of low-level human behavior are of significance to multiple research domains. In such settings, methods for explaining specific forecasts can enable domain experts to gain insights into the predictive relationships between behaviors. In this work, we introduce and address the following question: given a probabilistic forecasting model how can we identify observed windows that the model considers salient when making its forecasts? We build upon a general definition of information-theoretic saliency grounded in human perception and extend it to forecasting settings by leveraging a crucial attribute of the domain: a single observation can result in multiple valid futures. We propose to express the saliency of an observed window in terms of the differential entropy of the resulting predicted future distribution. In contrast to existing methods that either require explicit training of the saliency mechanism or access to the internal states of the forecasting model, we obtain a closed-form solution for the saliency map for commonly used density functions in probabilistic forecasting. We empirically demonstrate how our framework can recover salient observed windows from head pose features for the sample task of speaking-turn forecasting using a synthesized conversation dataset.Preprint. Under review.

show abstract

Socially and Contextually Aware Human Motion and Pose Forecasting

Cited by 2 publications

References 37 publications

Adversarial Attention for Human Motion Synthesis

Adversarial Attention for Human Motion Synthesis

Why Did This Model Forecast This Future? Closed-Form Temporal Saliency Towards Causal Explanations of Probabilistic Forecasts

Contact Info

Product

Resources

About