Which priors matter? Benchmarking models for learning latent dynamics

Abstract

Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving. In these settings, ML algorithms typically need to reason about a physical system using high dimensional observations, such as images, without access to the underlying state. Recently, several methods have proposed to integrate priors from classical mechanics into ML models to address the challenge of physical reasoning from images. In this work, we take a sober look at the current capabilities of these models. To this end, we introduce a suite consisting of 17 datasets with visual observations based on physical systems exhibiting a wide range of dynamics. We conduct a thorough and detailed comparison of the major classes of physically inspired methods alongside several strong baselines. While models that incorporate physical priors can often learn latent spaces with desirable properties, our results demonstrate that these methods fail to significantly improve upon standard techniques. Nonetheless, we find that the use of continuous and time-reversible dynamics benefits models of all classes.35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks.

Learning Trajectories of Hamiltonian Systems with Neural Networks

“…Addressing this limitation is an important line for future research with many promising results obtained recently (see, e.g., [4,6,7,14]). Another important research question is to identify the most efficient way to incorporate inductive biases from physics into modeling of dynamical systems, which is a topic of active debate at the moment [1,12].…”

“…Some symplectic integration schemes [5,9,25] make additional assumptions such as the separability of the Hamiltonian. 1 A recent model called Non-separable Symplectic Neural Networks (NSSNN) [28] releases this assumption by an improved symplectic integrator which works well for both separable and non-separable Hamiltonians.…”

A review of machine learning methods applied to structural dynamics and vibroacoustic

“…Elements of physics-guided loss and physics-guided architecture are used in neural ordinary differential equations (NODE) and energy-conserving neural networks. In NODEs, explicit integration steps are performed in each layer of the NN as one step evaluation of a standard ODE solver [442][443][444][445][446]. In energyconserving NN, the structure of Lagrangian and Hamiltonian equations have been embedded into the NN construction to ensure an energy-conservative behavior, as reviewed by Lutter and Peters [447] and implemented in different structures in [442,[448][449][450][451][452][453][454][455][456].…”

A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic

“…Elements of physics-guided loss and physics-guided architecture are used in Neural ordinary differential equations (NODE) and Energy-Conserving Neural Networks (ECNN). In NODEs, explicit integration steps are performed in each layer of the NN as one step evaluation of a common ODE solver [398][399][400][401][402]. In ECNN, the structure of Lagrangian and Hamiltonian equations have been embedded into the NN construction to ensure an energy conservative behavior, as reviewed by Lutter and Peters [403] and implemented in different structures in [398,[404][405][406][407][408][409][410][411][412].…”