Learning Structured Representations of Spatial and Interactive Dynamics for Trajectory Prediction in Crowded Scenes

Davchev, Todor; Burke, Michael G.; Ramamoorthy, Subramanian

doi:10.1109/lra.2020.3047778

Cited by 6 publications

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• struggle to discover the physical laws from data, e.g., output inadmissible solutions under spurious shifts [60]; • inefficient for knowledge transfer, e.g., require a large number of observations to adapt from one environment to another even if the underlying change is sparse [17]. These issues do not become any less severe with larger models [61].…”

Section: Spurious Shiftsmentioning

confidence: 99%

Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective

Liu¹,

Cadei²,

Schweizer³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning behavioral patterns from observational data has been a de-facto approach to motion forecasting. Yet, the current paradigm suffers from two shortcomings: brittle under covariate shift and inefficient for knowledge transfer. In this work, we propose to address these challenges from a causal representation perspective. We first introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables, namely invariant mechanisms, style confounders, and spurious features. We then introduce a learning framework that treats each group separately: (i) unlike the common practice of merging datasets collected from different locations, we exploit their subtle distinctions by means of an invariance loss encouraging the model to suppress spurious correlations; (ii) we devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph; (iii) we introduce a style consistency loss that not only enforces the structure of style representations but also serves as a selfsupervisory signal for test-time refinement on the fly. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations, outperforming prior state-of-the-art motion forecasting models for out-of-distribution generalization and lowshot transfer.

show abstract

Section: Spurious Shiftsmentioning

confidence: 99%